chore(i18n): sync translations with latest source changes (chunk 5/8, 92 changes)
This commit is contained in:
localizeflow[bot]
parent
abf99c481d
commit
b0d8fb988f
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,179 @@
|
|||
# 目标检测
|
||||
|
||||
我们之前处理的图像分类模型会接收一张图片并输出一个类别结果，例如在 MNIST 问题中输出类别“数字”。然而，在许多情况下，我们不仅仅想知道图片中有物体，还希望能够确定它们的具体位置。这正是**目标检测**的核心所在。
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/21)
|
||||
|
||||
|
||||
|
||||
> 图片来源：[YOLO v2 网站](https://pjreddie.com/darknet/yolov2/)
|
||||
|
||||
## 一种简单的目标检测方法
|
||||
|
||||
假设我们想在一张图片中找到一只猫，一种非常简单的目标检测方法如下：
|
||||
|
||||
1. 将图片分割成多个小块。
|
||||
2. 对每个小块进行图像分类。
|
||||
3. 对于分类结果激活值足够高的小块，可以认为其中包含目标物体。
|
||||
|
||||
|
||||
|
||||
> *图片来源：[练习笔记本](ObjectDetection-TF.ipynb)*
|
||||
|
||||
然而，这种方法远非理想，因为它只能非常粗略地定位物体的边界框。为了更精确地定位，我们需要进行某种**回归**来预测边界框的坐标——这需要特定的数据集。
|
||||
|
||||
## 用回归进行目标检测
|
||||
|
||||
[这篇博客文章](https://towardsdatascience.com/object-detection-with-neural-networks-a4e2c46b4491)对检测形状进行了很好的入门介绍。
|
||||
|
||||
## 目标检测数据集
|
||||
|
||||
在进行目标检测任务时，你可能会遇到以下数据集：
|
||||
|
||||
* [PASCAL VOC](http://host.robots.ox.ac.uk/pascal/VOC/) - 包含20个类别
|
||||
* [COCO](http://cocodataset.org/#home) - 常见物体上下文数据集。包含80个类别、边界框和分割掩码
|
||||
|
||||
|
||||
|
||||
## 目标检测评估指标
|
||||
|
||||
### 交并比（Intersection over Union）
|
||||
|
||||
对于图像分类来说，评估算法性能相对简单；但对于目标检测，我们需要同时评估类别的正确性以及推断出的边界框位置的精确性。后者通常使用**交并比**（IoU）来衡量两个框（或任意两个区域）的重叠程度。
|
||||
|
||||
|
||||
|
||||
> *图片来源：[这篇关于IoU的优秀博客文章](https://pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/)*
|
||||
|
||||
其思想很简单——将两个图形的交集面积除以它们的并集面积。对于两个完全相同的区域，IoU值为1；对于完全不相交的区域，IoU值为0。其他情况下，IoU值介于0到1之间。我们通常只考虑IoU值超过某个阈值的边界框。
|
||||
|
||||
### 平均精度（Average Precision）
|
||||
|
||||
假设我们想评估某个类别$C$的识别效果。为此，我们使用**平均精度**（AP）指标，其计算方法如下：
|
||||
|
||||
1. 绘制精度-召回曲线，显示检测阈值（从0到1）变化时的准确性。
|
||||
2. 根据阈值，我们会检测到图片中的更多或更少的物体，并得到不同的精度和召回值。
|
||||
3. 曲线如下所示：
|
||||
|
||||
<img src="https://github.com/shwars/NeuroWorkshop/raw/master/images/ObjDetectionPrecisionRecall.png"/>
|
||||
|
||||
> *图片来源：[NeuroWorkshop](http://github.com/shwars/NeuroWorkshop)*
|
||||
|
||||
类别$C$的平均精度是该曲线下的面积。更具体地说，召回轴通常分为10个部分，精度在这些点上取平均值：
|
||||
|
||||
$$
|
||||
AP = {1\over11}\sum_{i=0}^{10}\mbox{Precision}(\mbox{Recall}={i\over10})
|
||||
$$
|
||||
|
||||
### AP与IoU
|
||||
|
||||
我们只考虑IoU超过某个阈值的检测结果。例如，在PASCAL VOC数据集中，通常假设$\mbox{IoU Threshold} = 0.5$，而在COCO数据集中，AP会针对不同的$\mbox{IoU Threshold}$值进行测量。
|
||||
|
||||
<img src="https://github.com/shwars/NeuroWorkshop/raw/master/images/ObjDetectionPrecisionRecallIoU.png"/>
|
||||
|
||||
> *图片来源：[NeuroWorkshop](http://github.com/shwars/NeuroWorkshop)*
|
||||
|
||||
### 平均平均精度（Mean Average Precision - mAP）
|
||||
|
||||
目标检测的主要评估指标是**平均平均精度**（mAP）。它是所有类别的平均精度值，有时也会对$\mbox{IoU Threshold}$进行平均。更详细的mAP计算过程可以参考
|
||||
[这篇博客文章](https://medium.com/@timothycarlen/understanding-the-map-evaluation-metric-for-object-detection-a07fe6962cf3)，以及[这里的代码示例](https://gist.github.com/tarlen5/008809c3decf19313de216b9208f3734)。
|
||||
|
||||
## 不同的目标检测方法
|
||||
|
||||
目标检测算法主要分为两大类：
|
||||
|
||||
* **区域提议网络**（R-CNN, Fast R-CNN, Faster R-CNN）。其核心思想是生成**兴趣区域**（ROI），并对其运行CNN以寻找最大激活值。这种方法与简单方法有些类似，但ROI的生成方式更为智能。这类方法的主要缺点是速度较慢，因为需要对图像进行多次CNN分类器的处理。
|
||||
* **单次检测**（YOLO, SSD, RetinaNet）方法。这些架构设计的网络能够在一次处理过程中同时预测类别和ROI。
|
||||
|
||||
### R-CNN：基于区域的CNN
|
||||
|
||||
[R-CNN](http://islab.ulsan.ac.kr/files/announcement/513/rcnn_pami.pdf)使用[选择性搜索](http://www.huppelen.nl/publications/selectiveSearchDraft.pdf)生成ROI区域的层次结构，然后通过CNN特征提取器和SVM分类器确定物体类别，并通过线性回归确定*边界框*坐标。[官方论文](https://arxiv.org/pdf/1506.01497v1.pdf)
|
||||
|
||||
|
||||
|
||||
> *图片来源：van de Sande et al. ICCV’11*
|
||||
|
||||
|
||||
|
||||
> *图片来源：[这篇博客](https://towardsdatascience.com/r-cnn-fast-r-cnn-faster-r-cnn-yolo-object-detection-algorithms-36d53571365e)*
|
||||
|
||||
### F-RCNN - 快速R-CNN
|
||||
|
||||
这种方法与R-CNN类似，但区域是在应用卷积层之后定义的。
|
||||
|
||||
|
||||
|
||||
> 图片来源：[官方论文](https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Girshick_Fast_R-CNN_ICCV_2015_paper.pdf)，[arXiv](https://arxiv.org/pdf/1504.08083.pdf)，2015
|
||||
|
||||
### Faster R-CNN
|
||||
|
||||
这种方法的核心思想是使用神经网络预测ROI——即所谓的*区域提议网络*。[论文](https://arxiv.org/pdf/1506.01497.pdf)，2016
|
||||
|
||||
|
||||
|
||||
> 图片来源：[官方论文](https://arxiv.org/pdf/1506.01497.pdf)
|
||||
|
||||
### R-FCN：基于区域的全卷积网络
|
||||
|
||||
这种算法比Faster R-CNN更快。其核心思想如下：
|
||||
|
||||
1. 使用ResNet-101提取特征。
|
||||
2. 特征通过**位置敏感得分图**处理。每个类别$C$的物体被划分为$k\times k$区域，并训练预测物体的各部分。
|
||||
3. 对于$k\times k$区域中的每个部分，所有网络对物体类别进行投票，选择投票最多的类别。
|
||||
|
||||
|
||||
|
||||
> 图片来源：[官方论文](https://arxiv.org/abs/1605.06409)
|
||||
|
||||
### YOLO - 只看一次
|
||||
|
||||
YOLO是一种实时单次检测算法。其核心思想如下：
|
||||
|
||||
* 将图像划分为$S\times S$区域。
|
||||
* 对每个区域，**CNN**预测$n$个可能的物体、*边界框*坐标和*置信度*=*概率* * IoU。
|
||||
|
||||
|
||||
|
||||
> 图片来源：[官方论文](https://arxiv.org/abs/1506.02640)
|
||||
|
||||
### 其他算法
|
||||
|
||||
* RetinaNet: [官方论文](https://arxiv.org/abs/1708.02002)
|
||||
- [Torchvision中的PyTorch实现](https://pytorch.org/vision/stable/_modules/torchvision/models/detection/retinanet.html)
|
||||
- [Keras实现](https://github.com/fizyr/keras-retinanet)
|
||||
- [Keras示例中的RetinaNet目标检测](https://keras.io/examples/vision/retinanet/)
|
||||
* SSD（单次检测器）：[官方论文](https://arxiv.org/abs/1512.02325)
|
||||
|
||||
## ✍️ 练习：目标检测
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
[ObjectDetection.ipynb](ObjectDetection.ipynb)
|
||||
|
||||
## 总结
|
||||
|
||||
在本课中，你快速了解了实现目标检测的各种方法！
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
阅读以下关于YOLO的文章和笔记本，并尝试自己动手实践：
|
||||
|
||||
* [关于YOLO的优秀博客文章](https://www.analyticsvidhya.com/blog/2018/12/practical-guide-object-detection-yolo-framewor-python/)
|
||||
* [官方网站](https://pjreddie.com/darknet/yolo/)
|
||||
* YOLO：[Keras实现](https://github.com/experiencor/keras-yolo2)，[逐步讲解笔记本](https://github.com/experiencor/basic-yolo-keras/blob/master/Yolo%20Step-by-Step.ipynb)
|
||||
* YOLO v2：[Keras实现](https://github.com/experiencor/keras-yolo2)，[逐步讲解笔记本](https://github.com/experiencor/keras-yolo2/blob/master/Yolo%20Step-by-Step.ipynb)
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/22)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
* [目标检测](https://tjmachinelearning.com/lectures/1718/obj/) by Nikhil Sardana
|
||||
* [目标检测算法的优秀比较](https://lilianweng.github.io/lil-log/2018/12/27/object-detection-part-4.html)
|
||||
* [深度学习目标检测算法综述](https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852)
|
||||
* [目标检测算法基础入门](https://www.analyticsvidhya.com/blog/2018/10/a-step-by-step-introduction-to-the-basic-object-detection-algorithms-part-1/)
|
||||
* [Python实现Faster R-CNN进行目标检测](https://www.analyticsvidhya.com/blog/2018/11/implementation-faster-r-cnn-python-object-detection/)
|
||||
|
||||
## [作业：目标检测](lab/README.md)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
# 使用好莱坞头部数据集进行头部检测
|
||||
|
||||
来自 [AI 初学者课程](https://github.com/microsoft/ai-for-beginners) 的实验任务。
|
||||
|
||||
## 任务
|
||||
|
||||
通过视频监控摄像头流统计人数是一项重要任务，它可以帮助我们估算商店的访客数量、餐厅的繁忙时段等。为了解决这个任务，我们需要能够从不同角度检测人类的头部。为了训练一个能够检测人类头部的目标检测模型，我们可以使用 [好莱坞头部数据集](https://www.di.ens.fr/willow/research/headdetection/)。
|
||||
|
||||
## 数据集
|
||||
|
||||
[好莱坞头部数据集](https://www.di.ens.fr/willow/research/headdetection/release/HollywoodHeads.zip) 包含了 369,846 个标注的人类头部，分布在 224,740 帧好莱坞电影画面中。数据集采用 [https://host.robots.ox.ac.uk/pascal/VOC/](../../../../../../lessons/4-ComputerVision/11-ObjectDetection/lab/PASCAL VOC) 格式提供，每张图片都有一个对应的 XML 描述文件，格式如下：
|
||||
|
||||
```xml
|
||||
<annotation>
|
||||
<folder>HollywoodHeads</folder>
|
||||
<filename>mov_021_149390.jpeg</filename>
|
||||
<source>
|
||||
<database>HollywoodHeads 2015 Database</database>
|
||||
<annotation>HollywoodHeads 2015</annotation>
|
||||
<image>WILLOW</image>
|
||||
</source>
|
||||
<size>
|
||||
<width>608</width>
|
||||
<height>320</height>
|
||||
<depth>3</depth>
|
||||
</size>
|
||||
<segmented>0</segmented>
|
||||
<object>
|
||||
<name>head</name>
|
||||
<bndbox>
|
||||
<xmin>201</xmin>
|
||||
<ymin>1</ymin>
|
||||
<xmax>480</xmax>
|
||||
<ymax>263</ymax>
|
||||
</bndbox>
|
||||
<difficult>0</difficult>
|
||||
</object>
|
||||
<object>
|
||||
<name>head</name>
|
||||
<bndbox>
|
||||
<xmin>3</xmin>
|
||||
<ymin>4</ymin>
|
||||
<xmax>241</xmax>
|
||||
<ymax>285</ymax>
|
||||
</bndbox>
|
||||
<difficult>0</difficult>
|
||||
</object>
|
||||
</annotation>
|
||||
```
|
||||
|
||||
在这个数据集中，只有一个类别的对象 `head`，对于每个头部，都会提供其边界框的坐标。你可以使用 Python 库解析 XML，或者使用 [这个库](https://pypi.org/project/pascal-voc/) 直接处理 PASCAL VOC 格式。
|
||||
|
||||
## 训练目标检测模型
|
||||
|
||||
你可以通过以下方式之一来训练目标检测模型：
|
||||
|
||||
* 使用 [Azure Custom Vision](https://docs.microsoft.com/azure/cognitive-services/custom-vision-service/quickstarts/object-detection?tabs=visual-studio&WT.mc_id=academic-77998-cacaste) 及其 Python API，在云端以编程方式训练模型。Custom Vision 只能使用几百张图片进行模型训练，因此可能需要限制数据集的规模。
|
||||
* 使用 [Keras 教程](https://keras.io/examples/vision/retinanet/) 中的示例来训练 RetunaNet 模型。
|
||||
* 使用 [torchvision.models.detection.RetinaNet](https://pytorch.org/vision/stable/_modules/torchvision/models/detection/retinanet.html) 中的内置模块进行训练。
|
||||
|
||||
## 收获
|
||||
|
||||
目标检测是工业中经常需要完成的任务。虽然有一些服务可以用来执行目标检测（例如 [Azure Custom Vision](https://docs.microsoft.com/azure/cognitive-services/custom-vision-service/quickstarts/object-detection?tabs=visual-studio&WT.mc_id=academic-77998-cacaste)），但理解目标检测的工作原理并能够训练自己的模型是非常重要的。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。原始语言的文档应被视为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而产生的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
# 分割
|
||||
|
||||
我们之前学习了目标检测，它可以通过预测*边界框*来定位图像中的对象。然而，对于某些任务，我们不仅需要边界框，还需要更精确的对象定位。这项任务被称为**分割**。
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/23)
|
||||
|
||||
分割可以看作是**像素分类**，即对于图像中的**每个**像素，我们必须预测其类别（*背景*是其中一个类别）。主要有两种分割算法：
|
||||
|
||||
* **语义分割**只告诉像素的类别，不区分同一类别的不同对象。
|
||||
* **实例分割**将类别划分为不同的实例。
|
||||
|
||||
例如，对于实例分割，这些羊是不同的对象；而对于语义分割，所有的羊都被表示为一个类别。
|
||||
|
||||
<img src="../../../../../translated_images/zh-CN/instance_vs_semantic.eee9812bebf8cd45.webp" width="50%">
|
||||
|
||||
> 图片来源于[这篇博客文章](https://nirmalamurali.medium.com/image-classification-vs-semantic-segmentation-vs-instance-segmentation-625c33a08d50)
|
||||
|
||||
分割有不同的神经网络架构，但它们的结构都相同。从某种意义上说，它类似于你之前学习过的自动编码器，但不是解构原始图像，而是解构一个**掩码**。因此，分割网络包括以下部分：
|
||||
|
||||
* **编码器**从输入图像中提取特征
|
||||
* **解码器**将这些特征转换为**掩码图像**，其大小与通道数对应于类别数量。
|
||||
|
||||
<img src="../../../../../translated_images/zh-CN/segm.92442f2cb42ff4fa.webp" width="80%">
|
||||
|
||||
> 图片来源于[这篇论文](https://arxiv.org/pdf/2001.05566.pdf)
|
||||
|
||||
我们特别需要提到分割中使用的损失函数。在使用经典自动编码器时，我们需要测量两张图像之间的相似性，可以使用均方误差（MSE）来实现。在分割中，目标掩码图像中的每个像素表示类别编号（在第三维度上进行独热编码），因此我们需要使用特定于分类的损失函数——交叉熵损失，并对所有像素进行平均。如果掩码是二值的，则使用**二值交叉熵损失**（BCE）。
|
||||
|
||||
> ✅ 独热编码是一种将类别标签编码为长度等于类别数量的向量的方法。可以查看[这篇文章](https://datagy.io/sklearn-one-hot-encode/)了解这一技术。
|
||||
|
||||
## 医学影像中的分割
|
||||
|
||||
在本课中，我们将通过训练网络识别医学图像中的人类痣（也称为黑痣）来实际应用分割技术。我们将使用<a href="https://www.fc.up.pt/addi/ph2%20database.html">PH<sup>2</sup>数据库</a>的皮肤镜图像作为图像来源。该数据集包含200张图像，分为三类：典型痣、非典型痣和黑色素瘤。所有图像还包含一个对应的**掩码**，用于勾勒痣的轮廓。
|
||||
|
||||
> ✅ 这种技术特别适用于这种类型的医学影像，但你还能想到哪些其他的实际应用场景？
|
||||
|
||||
<img alt="navi" src="../../../../../translated_images/zh-CN/navi.2f20b727910110ea.webp"/>
|
||||
|
||||
> 图片来源于PH<sup>2</sup>数据库
|
||||
|
||||
我们将训练一个模型，将任何痣从背景中分割出来。
|
||||
|
||||
## ✍️ 练习：语义分割
|
||||
|
||||
打开以下笔记本，了解不同的语义分割架构，练习使用它们，并观察它们的实际效果。
|
||||
|
||||
* [语义分割 Pytorch](SemanticSegmentationPytorch.ipynb)
|
||||
* [语义分割 TensorFlow](SemanticSegmentationTF.ipynb)
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/24)
|
||||
|
||||
## 总结
|
||||
|
||||
分割是一种非常强大的图像分类技术，它超越了边界框，实现了像素级分类。这项技术在医学影像等领域有广泛应用。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
人体分割只是我们可以对人物图像进行的常见任务之一。其他重要任务包括**骨架检测**和**姿态检测**。试试[OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose)库，看看姿态检测的实际应用。
|
||||
|
||||
## 复习与自学
|
||||
|
||||
这篇[维基百科文章](https://wikipedia.org/wiki/Image_segmentation)提供了关于该技术各种应用的良好概述。自行学习实例分割和全景分割领域的相关知识。
|
||||
|
||||
## [作业](lab/README.md)
|
||||
|
||||
在本实验中，尝试使用[Segmentation Full Body MADS Dataset](https://www.kaggle.com/datasets/tapakah68/segmentation-full-body-mads-dataset)数据集进行**人体分割**。
|
||||
|
||||
---
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,22 @@
|
|||
# 人体分割
|
||||
|
||||
来自 [AI for Beginners Curriculum](https://github.com/microsoft/ai-for-beginners) 的实验任务。
|
||||
|
||||
## 任务
|
||||
|
||||
在视频制作中，例如天气预报，我们经常需要从摄像机中剪切出人像，并将其叠加到其他画面上。这通常通过 **抠像技术（chroma key）** 实现，即人在一个统一颜色的背景前拍摄，然后将背景移除。在本实验中，我们将训练一个神经网络模型来剪切出人像轮廓。
|
||||
|
||||
## 数据集
|
||||
|
||||
我们将使用来自 Kaggle 的 [Segmentation Full Body MADS Dataset](https://www.kaggle.com/datasets/tapakah68/segmentation-full-body-mads-dataset)。请从 Kaggle 手动下载该数据集。
|
||||
|
||||
## 起始笔记本
|
||||
|
||||
通过打开 [BodySegmentation.ipynb](../../../../../../lessons/4-ComputerVision/12-Segmentation/lab/BodySegmentation.ipynb) 开始实验。
|
||||
|
||||
## 收获
|
||||
|
||||
人体分割只是我们可以对人物图像进行的常见任务之一。其他重要任务包括 **骨架检测** 和 **姿态检测**。可以查看 [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) 库，了解如何实现这些任务。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用本翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
# 计算机视觉
|
||||
|
||||
|
||||
|
||||
在本节中，我们将学习以下内容：
|
||||
|
||||
* [计算机视觉和OpenCV简介](06-IntroCV/README.md)
|
||||
* [卷积神经网络](07-ConvNets/README.md)
|
||||
* [预训练网络和迁移学习](08-TransferLearning/README.md)
|
||||
* [自编码器](09-Autoencoders/README.md)
|
||||
* [生成对抗网络](10-GANs/README.md)
|
||||
* [目标检测](11-ObjectDetection/README.md)
|
||||
* [语义分割](12-Segmentation/README.md)
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用本翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
# 将文本表示为张量
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/25)
|
||||
|
||||
## 文本分类
|
||||
|
||||
在本节的第一部分中，我们将专注于**文本分类**任务。我们将使用 [AG News](https://www.kaggle.com/amananandrai/ag-news-classification-dataset) 数据集，该数据集包含如下新闻文章：
|
||||
|
||||
* 类别：科技
|
||||
* 标题：Ky.公司获得研究肽类的资助（美联社）
|
||||
* 正文：美联社 - 一家由路易斯维尔大学化学研究员创立的公司获得了一项开发资助...
|
||||
|
||||
我们的目标是根据文本将新闻项目分类到某个类别中。
|
||||
|
||||
## 表示文本
|
||||
|
||||
如果我们想用神经网络解决自然语言处理（NLP）任务，我们需要一种方法将文本表示为张量。计算机已经通过诸如 ASCII 或 UTF-8 等编码将文本字符表示为数字，这些数字映射到屏幕上的字体。
|
||||
|
||||
<img alt="显示字符映射到 ASCII 和二进制表示的图示" src="../../../../../translated_images/zh-CN/ascii-character-map.18ed6aa7f3b0a7ff.webp" width="50%"/>
|
||||
|
||||
> [图片来源](https://www.seobility.net/en/wiki/ASCII)
|
||||
|
||||
作为人类，我们理解每个字母**代表**什么，以及所有字符如何组合形成句子的单词。然而，计算机本身并没有这样的理解，神经网络需要在训练过程中学习这些含义。
|
||||
|
||||
因此，我们可以使用不同的方法来表示文本：
|
||||
|
||||
* **字符级表示**，即将文本中的每个字符表示为一个数字。假设我们的文本语料库中有 *C* 个不同的字符，那么单词 *Hello* 将被表示为一个 5x*C* 的张量。每个字母对应于一个独热编码的张量列。
|
||||
* **词级表示**，即我们创建一个包含文本中所有单词的**词汇表**，然后使用独热编码表示单词。这种方法更好一些，因为单个字母本身没有太多意义，因此通过使用更高级的语义概念——单词——我们简化了神经网络的任务。然而，由于词汇表的规模较大，我们需要处理高维稀疏张量。
|
||||
|
||||
无论采用哪种表示方式，我们首先需要将文本转换为**标记**序列，一个标记可以是字符、单词，甚至是单词的一部分。然后，我们使用**词汇表**将标记转换为数字，这个数字可以通过独热编码输入到神经网络中。
|
||||
|
||||
## N-Grams
|
||||
|
||||
在自然语言中，单词的确切含义只能在上下文中确定。例如，*神经网络* 和 *捕鱼网络* 的含义完全不同。考虑上下文的一种方法是基于单词对构建模型，并将单词对视为单独的词汇标记。这样，句子 *I like to go fishing* 将被表示为以下标记序列：*I like*，*like to*，*to go*，*go fishing*。这种方法的问题在于词汇表的规模显著增长，并且像 *go fishing* 和 *go shopping* 这样的组合被表示为不同的标记，尽管它们使用了相同的动词，但并没有共享任何语义相似性。
|
||||
|
||||
在某些情况下，我们可能会考虑使用三元组（tri-grams）——三个单词的组合。因此，这种方法通常被称为**n-grams**。此外，使用字符级表示时，n-grams 通常大致对应于不同的音节。
|
||||
|
||||
## 词袋模型和 TF/IDF
|
||||
|
||||
在解决文本分类等任务时，我们需要能够用一个固定大小的向量表示文本，这个向量将作为最终密集分类器的输入。最简单的方法之一是组合所有单个单词的表示，例如通过将它们相加。如果我们将每个单词的独热编码相加，我们将得到一个频率向量，显示每个单词在文本中出现的次数。这种文本表示被称为**词袋模型**（BoW）。
|
||||
|
||||
<img src="../../../../../translated_images/zh-CN/bow.3811869cff59368d.webp" width="90%"/>
|
||||
|
||||
> 图片由作者提供
|
||||
|
||||
词袋模型本质上表示了文本中出现了哪些单词以及它们的数量，这确实可以很好地指示文本的内容。例如，关于政治的新闻文章可能包含 *总统* 和 *国家* 等词，而科学出版物可能包含 *对撞机*、*发现* 等词。因此，单词频率在许多情况下可以很好地指示文本内容。
|
||||
|
||||
词袋模型的问题在于某些常见单词，例如 *and*、*is* 等，出现在大多数文本中，并且它们的频率最高，从而掩盖了真正重要的单词。我们可以通过考虑单词在整个文档集合中出现的频率来降低这些单词的重要性。这就是 TF/IDF 方法的主要思想，该方法在本课附带的笔记本中有更详细的介绍。
|
||||
|
||||
然而，这些方法都无法完全考虑文本的**语义**。我们需要更强大的神经网络模型来实现这一点，我们将在本节后续内容中讨论。
|
||||
|
||||
## ✍️ 练习：文本表示
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
* [使用 PyTorch 表示文本](TextRepresentationPyTorch.ipynb)
|
||||
* [使用 TensorFlow 表示文本](TextRepresentationTF.ipynb)
|
||||
|
||||
## 总结
|
||||
|
||||
到目前为止，我们已经学习了可以为不同单词添加频率权重的技术。然而，这些技术无法表示单词的含义或顺序。正如著名语言学家 J. R. Firth 在1935年所说：“单词的完整意义总是与上下文相关，任何脱离上下文的意义研究都不能被认真对待。”我们将在课程后续内容中学习如何通过语言建模从文本中捕获上下文信息。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
尝试使用词袋模型和不同的数据模型完成一些其他练习。你可以参考这个 [Kaggle 比赛](https://www.kaggle.com/competitions/word2vec-nlp-tutorial/overview/part-1-for-beginners-bag-of-words) 来获得灵感。
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/26)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
在 [Microsoft Learn](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-pytorch/?WT.mc_id=academic-77998-cacaste) 上练习你的文本嵌入和词袋模型技术。
|
||||
|
||||
## [作业：笔记本](assignment.md)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,577 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 文本分类任务\n",
|
||||
"\n",
|
||||
"如前所述，我们将专注于基于 **AG_NEWS** 数据集的简单文本分类任务，该任务是将新闻标题分类为以下四个类别之一：国际、体育、商业和科技。\n",
|
||||
"\n",
|
||||
"## 数据集\n",
|
||||
"\n",
|
||||
"此数据集已集成到 [`torchtext`](https://github.com/pytorch/text) 模块中，因此我们可以轻松访问它。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"import torchtext\n",
|
||||
"import os\n",
|
||||
"import collections\n",
|
||||
"os.makedirs('./data',exist_ok=True)\n",
|
||||
"train_dataset, test_dataset = torchtext.datasets.AG_NEWS(root='./data')\n",
|
||||
"classes = ['World', 'Sports', 'Business', 'Sci/Tech']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在这里，`train_dataset` 和 `test_dataset` 包含返回标签（类别编号）和文本对的集合，例如：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(3,\n",
|
||||
" \"Wall St. Bears Claw Back Into the Black (Reuters) Reuters - Short-sellers, Wall Street's dwindling\\\\band of ultra-cynics, are seeing green again.\")"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"list(train_dataset)[0]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"那么，让我们打印出数据集中前10条新标题：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"**Sci/Tech** -> Wall St. Bears Claw Back Into the Black (Reuters) Reuters - Short-sellers, Wall Street's dwindling\\band of ultra-cynics, are seeing green again.\n",
|
||||
"**Sci/Tech** -> Carlyle Looks Toward Commercial Aerospace (Reuters) Reuters - Private investment firm Carlyle Group,\\which has a reputation for making well-timed and occasionally\\controversial plays in the defense industry, has quietly placed\\its bets on another part of the market.\n",
|
||||
"**Sci/Tech** -> Oil and Economy Cloud Stocks' Outlook (Reuters) Reuters - Soaring crude prices plus worries\\about the economy and the outlook for earnings are expected to\\hang over the stock market next week during the depth of the\\summer doldrums.\n",
|
||||
"**Sci/Tech** -> Iraq Halts Oil Exports from Main Southern Pipeline (Reuters) Reuters - Authorities have halted oil export\\flows from the main pipeline in southern Iraq after\\intelligence showed a rebel militia could strike\\infrastructure, an oil official said on Saturday.\n",
|
||||
"**Sci/Tech** -> Oil prices soar to all-time record, posing new menace to US economy (AFP) AFP - Tearaway world oil prices, toppling records and straining wallets, present a new economic menace barely three months before the US presidential elections.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for i,x in zip(range(5),train_dataset):\n",
|
||||
" print(f\"**{classes[x[0]]}** -> {x[1]}\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"因为数据集是迭代器，如果我们想多次使用数据，则需要将其转换为列表：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"train_dataset, test_dataset = torchtext.datasets.AG_NEWS(root='./data')\n",
|
||||
"train_dataset = list(train_dataset)\n",
|
||||
"test_dataset = list(test_dataset)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 分词\n",
|
||||
"\n",
|
||||
"现在我们需要将文本转换为可以表示为张量的**数字**。如果我们想要基于单词的表示，需要完成两件事：\n",
|
||||
"* 使用**分词器**将文本拆分为**标记**\n",
|
||||
"* 构建这些标记的**词汇表**。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['he', 'said', 'hello']"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"tokenizer = torchtext.data.utils.get_tokenizer('basic_english')\n",
|
||||
"tokenizer('He said: hello')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"counter = collections.Counter()\n",
|
||||
"for (label, line) in train_dataset:\n",
|
||||
" counter.update(tokenizer(line))\n",
|
||||
"vocab = torchtext.vocab.vocab(counter, min_freq=1)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"使用词汇表，我们可以轻松地将标记化的字符串编码为一组数字：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 19,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Vocab size if 95810\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[599, 3279, 97, 1220, 329, 225, 7368]"
|
||||
]
|
||||
},
|
||||
"execution_count": 19,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vocab_size = len(vocab)\n",
|
||||
"print(f\"Vocab size if {vocab_size}\")\n",
|
||||
"\n",
|
||||
"stoi = vocab.get_stoi() # dict to convert tokens to indices\n",
|
||||
"\n",
|
||||
"def encode(x):\n",
|
||||
" return [stoi[s] for s in tokenizer(x)]\n",
|
||||
"\n",
|
||||
"encode('I love to play with my words')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 词袋文本表示法\n",
|
||||
"\n",
|
||||
"由于单词代表了意义，有时我们可以通过仅仅查看单个单词，而不考虑它们在句子中的顺序，就能理解一段文本的含义。例如，在分类新闻时，像 *weather*（天气）、*snow*（雪）这样的单词可能表明是 *天气预报*，而像 *stocks*（股票）、*dollar*（美元）这样的单词则可能属于 *财经新闻*。\n",
|
||||
"\n",
|
||||
"**词袋**（Bag of Words, BoW）向量表示法是最常用的传统向量表示法。每个单词都与一个向量索引相关联，向量的元素包含某个单词在给定文档中出现的次数。\n",
|
||||
"\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"> **Note**: 你也可以将 BoW 理解为文本中每个单词的独热编码向量的总和。\n",
|
||||
"\n",
|
||||
"下面是一个使用 Scikit Learn Python 库生成词袋表示的示例：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([[1, 1, 0, 2, 0, 0, 0, 0, 0]], dtype=int64)"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.feature_extraction.text import CountVectorizer\n",
|
||||
"vectorizer = CountVectorizer()\n",
|
||||
"corpus = [\n",
|
||||
" 'I like hot dogs.',\n",
|
||||
" 'The dog ran fast.',\n",
|
||||
" 'Its hot outside.',\n",
|
||||
" ]\n",
|
||||
"vectorizer.fit_transform(corpus)\n",
|
||||
"vectorizer.transform(['My dog likes hot dogs on a hot day.']).toarray()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"要从我们的 AG_NEWS 数据集的向量表示计算词袋向量，可以使用以下函数：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 20,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"tensor([2., 1., 2., ..., 0., 0., 0.])\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vocab_size = len(vocab)\n",
|
||||
"\n",
|
||||
"def to_bow(text,bow_vocab_size=vocab_size):\n",
|
||||
" res = torch.zeros(bow_vocab_size,dtype=torch.float32)\n",
|
||||
" for i in encode(text):\n",
|
||||
" if i<bow_vocab_size:\n",
|
||||
" res[i] += 1\n",
|
||||
" return res\n",
|
||||
"\n",
|
||||
"print(to_bow(train_dataset[0][1]))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意：** 这里我们使用全局变量 `vocab_size` 来指定默认的词汇表大小。由于词汇表的大小通常非常大，我们可以将词汇表的大小限制为最常用的词汇。尝试降低 `vocab_size` 的值并运行下面的代码，看看它如何影响准确性。你应该预期会有一些准确性的下降，但不会太剧烈，以换取更高的性能。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 训练 BoW 分类器\n",
|
||||
"\n",
|
||||
"现在我们已经学习了如何构建文本的词袋表示，让我们在其基础上训练一个分类器。首先，我们需要将数据集转换为适合训练的形式，即将所有位置向量表示转换为词袋表示。这可以通过将 `bowify` 函数作为标准 torch `DataLoader` 的 `collate_fn` 参数传递来实现：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 21,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from torch.utils.data import DataLoader\n",
|
||||
"import numpy as np \n",
|
||||
"\n",
|
||||
"# this collate function gets list of batch_size tuples, and needs to \n",
|
||||
"# return a pair of label-feature tensors for the whole minibatch\n",
|
||||
"def bowify(b):\n",
|
||||
" return (\n",
|
||||
" torch.LongTensor([t[0]-1 for t in b]),\n",
|
||||
" torch.stack([to_bow(t[1]) for t in b])\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"train_loader = DataLoader(train_dataset, batch_size=16, collate_fn=bowify, shuffle=True)\n",
|
||||
"test_loader = DataLoader(test_dataset, batch_size=16, collate_fn=bowify, shuffle=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在让我们定义一个简单的分类器神经网络，它包含一个线性层。输入向量的大小等于 `vocab_size`，输出大小对应于类别的数量（4）。由于我们正在解决分类任务，最终的激活函数是 `LogSoftmax()`。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 22,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"net = torch.nn.Sequential(torch.nn.Linear(vocab_size,4),torch.nn.LogSoftmax(dim=1))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在我们将定义标准的PyTorch训练循环。由于我们的数据集相当大，为了教学目的，我们将只训练一个周期，有时甚至少于一个周期（通过指定`epoch_size`参数可以限制训练）。我们还将在训练过程中报告累计的训练准确率；报告的频率通过`report_freq`参数指定。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 24,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def train_epoch(net,dataloader,lr=0.01,optimizer=None,loss_fn = torch.nn.NLLLoss(),epoch_size=None, report_freq=200):\n",
|
||||
" optimizer = optimizer or torch.optim.Adam(net.parameters(),lr=lr)\n",
|
||||
" net.train()\n",
|
||||
" total_loss,acc,count,i = 0,0,0,0\n",
|
||||
" for labels,features in dataloader:\n",
|
||||
" optimizer.zero_grad()\n",
|
||||
" out = net(features)\n",
|
||||
" loss = loss_fn(out,labels) #cross_entropy(out,labels)\n",
|
||||
" loss.backward()\n",
|
||||
" optimizer.step()\n",
|
||||
" total_loss+=loss\n",
|
||||
" _,predicted = torch.max(out,1)\n",
|
||||
" acc+=(predicted==labels).sum()\n",
|
||||
" count+=len(labels)\n",
|
||||
" i+=1\n",
|
||||
" if i%report_freq==0:\n",
|
||||
" print(f\"{count}: acc={acc.item()/count}\")\n",
|
||||
" if epoch_size and count>epoch_size:\n",
|
||||
" break\n",
|
||||
" return total_loss.item()/count, acc.item()/count"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 25,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3200: acc=0.8028125\n",
|
||||
"6400: acc=0.8371875\n",
|
||||
"9600: acc=0.8534375\n",
|
||||
"12800: acc=0.85765625\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(0.026090790722161722, 0.8620069296375267)"
|
||||
]
|
||||
},
|
||||
"execution_count": 25,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"train_epoch(net,train_loader,epoch_size=15000)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 二元组、三元组和N元组\n",
|
||||
"\n",
|
||||
"词袋方法的一个局限性是，有些词是多词表达的一部分。例如，“热狗”这个词的含义与“热”和“狗”在其他语境中的含义完全不同。如果我们总是用相同的向量表示“热”和“狗”，可能会让模型感到困惑。\n",
|
||||
"\n",
|
||||
"为了解决这个问题，**N元组表示**通常用于文档分类方法中，其中每个单词、双词或三词的频率是训练分类器的有用特征。例如，在二元组表示中，我们会将所有的词对添加到词汇表中，除了原始单词之外。\n",
|
||||
"\n",
|
||||
"下面是一个使用Scikit Learn生成二元组词袋表示的示例：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 26,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Vocabulary:\n",
|
||||
" {'i': 7, 'like': 11, 'hot': 4, 'dogs': 2, 'i like': 8, 'like hot': 12, 'hot dogs': 5, 'the': 16, 'dog': 0, 'ran': 14, 'fast': 3, 'the dog': 17, 'dog ran': 1, 'ran fast': 15, 'its': 9, 'outside': 13, 'its hot': 10, 'hot outside': 6}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([[1, 0, 1, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],\n",
|
||||
" dtype=int64)"
|
||||
]
|
||||
},
|
||||
"execution_count": 26,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"bigram_vectorizer = CountVectorizer(ngram_range=(1, 2), token_pattern=r'\\b\\w+\\b', min_df=1)\n",
|
||||
"corpus = [\n",
|
||||
" 'I like hot dogs.',\n",
|
||||
" 'The dog ran fast.',\n",
|
||||
" 'Its hot outside.',\n",
|
||||
" ]\n",
|
||||
"bigram_vectorizer.fit_transform(corpus)\n",
|
||||
"print(\"Vocabulary:\\n\",bigram_vectorizer.vocabulary_)\n",
|
||||
"bigram_vectorizer.transform(['My dog likes hot dogs on a hot day.']).toarray()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"N-gram方法的主要缺点是词汇量会迅速增长。在实际应用中，我们需要将N-gram表示与一些降维技术结合使用，例如*嵌入*，我们将在下一单元中讨论。\n",
|
||||
"\n",
|
||||
"为了在我们的**AG News**数据集中使用N-gram表示，我们需要构建一个特殊的N-gram词汇：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 27,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Bigram vocabulary length = 1308842\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"counter = collections.Counter()\n",
|
||||
"for (label, line) in train_dataset:\n",
|
||||
" l = tokenizer(line)\n",
|
||||
" counter.update(torchtext.data.utils.ngrams_iterator(l,ngrams=2))\n",
|
||||
" \n",
|
||||
"bi_vocab = torchtext.vocab.vocab(counter, min_freq=1)\n",
|
||||
"\n",
|
||||
"print(\"Bigram vocabulary length = \",len(bi_vocab))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们可以使用上述相同的代码来训练分类器，但这样会非常占用内存。在下一节中，我们将使用嵌入来训练二元分类器。\n",
|
||||
"\n",
|
||||
"> **注意:** 你只能保留那些在文本中出现次数超过指定数量的n元组。这将确保不常见的二元组被省略，并显著减少维度。为此，可以将`min_freq`参数设置为更高的值，并观察词汇表长度的变化。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 词频-逆文档频率 TF-IDF\n",
|
||||
"\n",
|
||||
"在 BoW 表示法中，单词的出现被均等对待，无论单词本身如何。然而，很明显，像 *a*、*in* 这样的常见词对于分类的作用远不如一些专业术语重要。实际上，在大多数 NLP 任务中，有些词比其他词更相关。\n",
|
||||
"\n",
|
||||
"**TF-IDF** 是 **词频-逆文档频率** 的缩写。它是袋子模型的一种变体，其中不是用二进制的 0/1 值来表示单词是否出现在文档中，而是使用一个浮点值，该值与单词在语料库中的出现频率相关。\n",
|
||||
"\n",
|
||||
"更正式地说，单词 $i$ 在文档 $j$ 中的权重 $w_{ij}$ 定义为：\n",
|
||||
"$$\n",
|
||||
"w_{ij} = tf_{ij}\\times\\log({N\\over df_i})\n",
|
||||
"$$\n",
|
||||
"其中：\n",
|
||||
"* $tf_{ij}$ 是单词 $i$ 在文档 $j$ 中出现的次数，即我们之前看到的 BoW 值\n",
|
||||
"* $N$ 是语料库中的文档总数\n",
|
||||
"* $df_i$ 是整个语料库中包含单词 $i$ 的文档数量\n",
|
||||
"\n",
|
||||
"TF-IDF 值 $w_{ij}$ 随着单词在文档中出现的次数成比例增加，同时会受到语料库中包含该单词的文档数量的影响，从而调整某些单词出现频率较高的情况。例如，如果某个单词出现在语料库中的*每一篇*文档中，那么 $df_i=N$，且 $w_{ij}=0$，这些词将被完全忽略。\n",
|
||||
"\n",
|
||||
"你可以使用 Scikit Learn 轻松实现文本的 TF-IDF 向量化：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 28,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([[0.43381609, 0. , 0.43381609, 0. , 0.65985664,\n",
|
||||
" 0.43381609, 0. , 0. , 0. , 0. ,\n",
|
||||
" 0. , 0. , 0. , 0. , 0. ,\n",
|
||||
" 0. ]])"
|
||||
]
|
||||
},
|
||||
"execution_count": 28,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.feature_extraction.text import TfidfVectorizer\n",
|
||||
"vectorizer = TfidfVectorizer(ngram_range=(1,2))\n",
|
||||
"vectorizer.fit_transform(corpus)\n",
|
||||
"vectorizer.transform(['My dog likes hot dogs on a hot day.']).toarray()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 结论\n",
|
||||
"\n",
|
||||
"尽管TF-IDF表示法为不同的单词提供了频率权重，但它无法表达意义或顺序。正如著名语言学家J.R. Firth在1935年所说：“一个词的完整意义总是与上下文相关，任何脱离上下文的意义研究都不应被认真对待。” 在课程的后续部分，我们将学习如何通过语言建模从文本中捕捉上下文信息。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用本翻译而引起的任何误解或误读不承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.12 ('py38')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "7b9040985e748e4e2d4c689892456ad7",
|
||||
"translation_date": "2025-08-31T10:58:15+00:00",
|
||||
"source_file": "lessons/5-NLP/13-TextRep/TextRepresentationPyTorch.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,647 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 文本分类任务\n",
|
||||
"\n",
|
||||
"在本模块中，我们将从一个简单的文本分类任务开始，基于 **[AG_NEWS](http://www.di.unipi.it/~gulli/AG_corpus_of_news_articles.html)** 数据集：我们将把新闻标题分类为以下四个类别之一：国际、体育、商业和科技。\n",
|
||||
"\n",
|
||||
"## 数据集\n",
|
||||
"\n",
|
||||
"为了加载数据集，我们将使用 **[TensorFlow Datasets](https://www.tensorflow.org/datasets)** API。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorflow import keras\n",
|
||||
"import tensorflow_datasets as tfds\n",
|
||||
"\n",
|
||||
"# In this tutorial, we will be training a lot of models. In order to use GPU memory cautiously,\n",
|
||||
"# we will set tensorflow option to grow GPU memory allocation when required.\n",
|
||||
"physical_devices = tf.config.list_physical_devices('GPU') \n",
|
||||
"if len(physical_devices)>0:\n",
|
||||
" tf.config.experimental.set_memory_growth(physical_devices[0], True)\n",
|
||||
"\n",
|
||||
"dataset = tfds.load('ag_news_subset')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们现在可以通过使用 `dataset['train']` 和 `dataset['test']` 分别访问数据集的训练和测试部分：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Length of train dataset = 120000\n",
|
||||
"Length of test dataset = 7600\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"ds_train = dataset['train']\n",
|
||||
"ds_test = dataset['test']\n",
|
||||
"\n",
|
||||
"print(f\"Length of train dataset = {len(ds_train)}\")\n",
|
||||
"print(f\"Length of test dataset = {len(ds_test)}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"让我们打印出数据集中前10条新标题：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3 (Sci/Tech) -> b'AMD Debuts Dual-Core Opteron Processor' b'AMD #39;s new dual-core Opteron chip is designed mainly for corporate computing applications, including databases, Web services, and financial transactions.'\n",
|
||||
"1 (Sports) -> b\"Wood's Suspension Upheld (Reuters)\" b'Reuters - Major League Baseball\\\\Monday announced a decision on the appeal filed by Chicago Cubs\\\\pitcher Kerry Wood regarding a suspension stemming from an\\\\incident earlier this season.'\n",
|
||||
"2 (Business) -> b'Bush reform may have blue states seeing red' b'President Bush #39;s quot;revenue-neutral quot; tax reform needs losers to balance its winners, and people claiming the federal deduction for state and local taxes may be in administration planners #39; sights, news reports say.'\n",
|
||||
"3 (Sci/Tech) -> b\"'Halt science decline in schools'\" b'Britain will run out of leading scientists unless science education is improved, says Professor Colin Pillinger.'\n",
|
||||
"1 (Sports) -> b'Gerrard leaves practice' b'London, England (Sports Network) - England midfielder Steven Gerrard injured his groin late in Thursday #39;s training session, but is hopeful he will be ready for Saturday #39;s World Cup qualifier against Austria.'\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"classes = ['World', 'Sports', 'Business', 'Sci/Tech']\n",
|
||||
"\n",
|
||||
"for i,x in zip(range(5),ds_train):\n",
|
||||
" print(f\"{x['label']} ({classes[x['label']]}) -> {x['title']} {x['description']}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 文本向量化\n",
|
||||
"\n",
|
||||
"现在我们需要将文本转换为可以表示为张量的**数字**。如果我们想要词级别的表示，需要完成以下两件事：\n",
|
||||
"\n",
|
||||
"* 使用一个**分词器**将文本拆分为**标记**。\n",
|
||||
"* 构建这些标记的**词汇表**。\n",
|
||||
"\n",
|
||||
"### 限制词汇表大小\n",
|
||||
"\n",
|
||||
"在 AG News 数据集的例子中，词汇表的大小相当大，超过了 10 万个单词。一般来说，我们不需要那些在文本中很少出现的单词——只有少数句子会包含它们，而模型也无法从中学习。因此，通过向向量化器构造函数传递一个参数，将词汇表大小限制为一个较小的数字是合理的：\n",
|
||||
"\n",
|
||||
"以上两个步骤都可以通过 **TextVectorization** 层来处理。我们可以实例化一个向量化器对象，然后调用 `adapt` 方法来遍历所有文本并构建词汇表：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"vocab_size = 50000\n",
|
||||
"vectorizer = keras.layers.experimental.preprocessing.TextVectorization(max_tokens=vocab_size)\n",
|
||||
"vectorizer.adapt(ds_train.take(500).map(lambda x: x['title']+' '+x['description']))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意** 我们仅使用整个数据集的一部分来构建词汇表。这样做是为了加快执行速度，避免让您等待太久。然而，这样做存在一定风险，即整个数据集中的某些词可能不会被包含在词汇表中，并在训练过程中被忽略。因此，使用完整的词汇表大小并在`adapt`过程中遍历整个数据集应该可以提高最终的准确性，但提升幅度不会太大。\n",
|
||||
"\n",
|
||||
"现在我们可以访问实际的词汇表：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"['', '[UNK]', 'the', 'to', 'a', 'in', 'of', 'and', 'on', 'for']\n",
|
||||
"Length of vocabulary: 5335\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vocab = vectorizer.get_vocabulary()\n",
|
||||
"vocab_size = len(vocab)\n",
|
||||
"print(vocab[:10])\n",
|
||||
"print(f\"Length of vocabulary: {vocab_size}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"使用向量化器，我们可以轻松地将任何文本编码为一组数字：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tf.Tensor: shape=(7,), dtype=int64, numpy=array([ 112, 3695, 3, 304, 11, 1041, 1], dtype=int64)>"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vectorizer('I love to play with my words')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 词袋文本表示法\n",
|
||||
"\n",
|
||||
"由于单词承载着意义，有时我们可以通过单独查看单词，而不考虑它们在句子中的顺序，来理解一段文本的含义。例如，在新闻分类中，像 *天气* 和 *雪* 这样的词可能表明 *天气预报*，而像 *股票* 和 *美元* 这样的词则可能属于 *财经新闻*。\n",
|
||||
"\n",
|
||||
"**词袋** (BoW) 向量表示法是最容易理解的传统向量表示法。每个单词都与一个向量索引相关联，向量中的元素表示某个文档中每个单词出现的次数。\n",
|
||||
"\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"> **注意**: 你也可以将 BoW 理解为文本中每个单词的独热编码向量的总和。\n",
|
||||
"\n",
|
||||
"下面是一个使用 Scikit Learn Python 库生成词袋表示法的示例：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([[1, 1, 0, 2, 0, 0, 0, 0, 0]], dtype=int64)"
|
||||
]
|
||||
},
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.feature_extraction.text import CountVectorizer\n",
|
||||
"sc_vectorizer = CountVectorizer()\n",
|
||||
"corpus = [\n",
|
||||
" 'I like hot dogs.',\n",
|
||||
" 'The dog ran fast.',\n",
|
||||
" 'Its hot outside.',\n",
|
||||
" ]\n",
|
||||
"sc_vectorizer.fit_transform(corpus)\n",
|
||||
"sc_vectorizer.transform(['My dog likes hot dogs on a hot day.']).toarray()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们还可以使用上面定义的Keras向量化器，将每个单词编号转换为独热编码，并将所有这些向量相加：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([0., 5., 0., ..., 0., 0., 0.], dtype=float32)"
|
||||
]
|
||||
},
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def to_bow(text):\n",
|
||||
" return tf.reduce_sum(tf.one_hot(vectorizer(text),vocab_size),axis=0)\n",
|
||||
"\n",
|
||||
"to_bow('My dog likes hot dogs on a hot day.').numpy()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意**：您可能会惊讶地发现结果与之前的示例不同。原因在于，在 Keras 示例中，向量的长度对应于词汇表的大小，而该词汇表是基于整个 AG News 数据集构建的；而在 Scikit Learn 示例中，我们是根据示例文本动态构建词汇表的。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 训练 BoW 分类器\n",
|
||||
"\n",
|
||||
"现在我们已经了解了如何构建文本的词袋表示，让我们训练一个使用它的分类器。首先，我们需要将数据集转换为词袋表示。这可以通过以下方式使用 `map` 函数来实现：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"batch_size = 128\n",
|
||||
"\n",
|
||||
"ds_train_bow = ds_train.map(lambda x: (to_bow(x['title']+x['description']),x['label'])).batch(batch_size)\n",
|
||||
"ds_test_bow = ds_test.map(lambda x: (to_bow(x['title']+x['description']),x['label'])).batch(batch_size)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在让我们定义一个简单的分类器神经网络，它包含一个线性层。输入大小是`vocab_size`，输出大小对应于类别数量（4）。由于我们正在解决一个分类任务，最终的激活函数是**softmax**：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"938/938 [==============================] - 66s 70ms/step - loss: 0.6144 - acc: 0.8427 - val_loss: 0.4416 - val_acc: 0.8697\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<keras.callbacks.History at 0x20c70a947f0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = keras.models.Sequential([\n",
|
||||
" keras.layers.Dense(4,activation='softmax',input_shape=(vocab_size,))\n",
|
||||
"])\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['acc'])\n",
|
||||
"model.fit(ds_train_bow,validation_data=ds_test_bow)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"由于我们有4个类别，超过80%的准确率是一个不错的结果。\n",
|
||||
"\n",
|
||||
"## 将分类器训练为一个网络\n",
|
||||
"\n",
|
||||
"因为向量化器也是一个 Keras 层，我们可以定义一个包含它的网络，并进行端到端的训练。这样我们就不需要使用 `map` 来向量化数据集，只需将原始数据集传递到网络的输入即可。\n",
|
||||
"\n",
|
||||
"> **注意**: 我们仍然需要对数据集应用映射操作，以将字段从字典（例如 `title`、`description` 和 `label`）转换为元组。然而，当从磁盘加载数据时，我们可以一开始就构建一个具有所需结构的数据集。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"model\"\n",
|
||||
"_________________________________________________________________\n",
|
||||
" Layer (type) Output Shape Param # \n",
|
||||
"=================================================================\n",
|
||||
" input_1 (InputLayer) [(None, 1)] 0 \n",
|
||||
" \n",
|
||||
" text_vectorization (TextVec (None, None) 0 \n",
|
||||
" torization) \n",
|
||||
" \n",
|
||||
" tf.one_hot (TFOpLambda) (None, None, 5335) 0 \n",
|
||||
" \n",
|
||||
" tf.math.reduce_sum (TFOpLam (None, 5335) 0 \n",
|
||||
" bda) \n",
|
||||
" \n",
|
||||
" dense_2 (Dense) (None, 4) 21344 \n",
|
||||
" \n",
|
||||
"=================================================================\n",
|
||||
"Total params: 21,344\n",
|
||||
"Trainable params: 21,344\n",
|
||||
"Non-trainable params: 0\n",
|
||||
"_________________________________________________________________\n",
|
||||
"938/938 [==============================] - 73s 77ms/step - loss: 0.6057 - acc: 0.8414 - val_loss: 0.4202 - val_acc: 0.8736\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<keras.callbacks.History at 0x20c721521f0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def extract_text(x):\n",
|
||||
" return x['title']+' '+x['description']\n",
|
||||
"\n",
|
||||
"def tupelize(x):\n",
|
||||
" return (extract_text(x),x['label'])\n",
|
||||
"\n",
|
||||
"inp = keras.Input(shape=(1,),dtype=tf.string)\n",
|
||||
"x = vectorizer(inp)\n",
|
||||
"x = tf.reduce_sum(tf.one_hot(x,vocab_size),axis=1)\n",
|
||||
"out = keras.layers.Dense(4,activation='softmax')(x)\n",
|
||||
"model = keras.models.Model(inp,out)\n",
|
||||
"model.summary()\n",
|
||||
"\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['acc'])\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(batch_size),validation_data=ds_test.map(tupelize).batch(batch_size))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 二元组、三元组和 n 元组\n",
|
||||
"\n",
|
||||
"词袋模型的一个局限性在于，有些词是多词表达的一部分。例如，“hot dog”这个词的含义与单独的“hot”和“dog”在其他语境中的含义完全不同。如果我们始终用相同的向量表示“hot”和“dog”，可能会让模型感到困惑。\n",
|
||||
"\n",
|
||||
"为了解决这个问题，**n 元组表示法**常被用于文档分类方法中，其中每个单词、双词组或三词组的频率是训练分类器的一个有用特征。例如，在二元组表示中，我们会将所有的单词对添加到词汇表中，除了原始单词之外。\n",
|
||||
"\n",
|
||||
"以下是一个使用 Scikit Learn 生成二元组词袋表示的示例：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Vocabulary:\n",
|
||||
" {'i': 7, 'like': 11, 'hot': 4, 'dogs': 2, 'i like': 8, 'like hot': 12, 'hot dogs': 5, 'the': 16, 'dog': 0, 'ran': 14, 'fast': 3, 'the dog': 17, 'dog ran': 1, 'ran fast': 15, 'its': 9, 'outside': 13, 'its hot': 10, 'hot outside': 6}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([[1, 0, 1, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],\n",
|
||||
" dtype=int64)"
|
||||
]
|
||||
},
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"bigram_vectorizer = CountVectorizer(ngram_range=(1, 2), token_pattern=r'\\b\\w+\\b', min_df=1)\n",
|
||||
"corpus = [\n",
|
||||
" 'I like hot dogs.',\n",
|
||||
" 'The dog ran fast.',\n",
|
||||
" 'Its hot outside.',\n",
|
||||
" ]\n",
|
||||
"bigram_vectorizer.fit_transform(corpus)\n",
|
||||
"print(\"Vocabulary:\\n\",bigram_vectorizer.vocabulary_)\n",
|
||||
"bigram_vectorizer.transform(['My dog likes hot dogs on a hot day.']).toarray()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"n-gram 方法的主要缺点是词汇表的大小会迅速增长。在实际应用中，我们需要将 n-gram 表示与降维技术（例如 *embeddings*）结合使用，这将在下一单元中讨论。\n",
|
||||
"\n",
|
||||
"为了在我们的 **AG News** 数据集中使用 n-gram 表示，我们需要将 `ngrams` 参数传递给 `TextVectorization` 构造函数。二元组词汇表的长度会**显著增加**，在我们的例子中，它超过了 130 万个标记！因此，将二元组标记限制在一个合理的数量范围内是有意义的。\n",
|
||||
"\n",
|
||||
"我们可以使用与上面相同的代码来训练分类器，但这样做会非常占用内存。在下一单元中，我们将使用 embeddings 来训练二元组分类器。同时，你可以在这个笔记本中尝试训练二元组分类器，看看是否能获得更高的准确率。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 自动计算 BoW 向量\n",
|
||||
"\n",
|
||||
"在上面的例子中，我们通过手动将单个词的一次性编码相加来计算 BoW 向量。然而，最新版本的 TensorFlow 允许我们通过向向量化器构造函数传递 `output_mode='count` 参数来自动计算 BoW 向量。这使得定义和训练我们的模型变得更加简单：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training vectorizer\n",
|
||||
"938/938 [==============================] - 7s 7ms/step - loss: 0.5929 - acc: 0.8486 - val_loss: 0.4168 - val_acc: 0.8772\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<keras.callbacks.History at 0x20c725217c0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = keras.models.Sequential([\n",
|
||||
" keras.layers.experimental.preprocessing.TextVectorization(max_tokens=vocab_size,output_mode='count'),\n",
|
||||
" keras.layers.Dense(4,input_shape=(vocab_size,), activation='softmax')\n",
|
||||
"])\n",
|
||||
"print(\"Training vectorizer\")\n",
|
||||
"model.layers[0].adapt(ds_train.take(500).map(extract_text))\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['acc'])\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(batch_size),validation_data=ds_test.map(tupelize).batch(batch_size))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 词频-逆文档频率 (TF-IDF)\n",
|
||||
"\n",
|
||||
"在 BoW 表示法中，无论单词本身如何，单词出现的权重都使用相同的技术进行计算。然而，很明显，像 *a* 和 *in* 这样的高频词对于分类的意义远不如一些专业术语。在大多数 NLP 任务中，有些词比其他词更重要。\n",
|
||||
"\n",
|
||||
"**TF-IDF** 是 **词频-逆文档频率** 的缩写。它是袋子模型（bag-of-words）的一个变体，其中不是用二进制的 0/1 值来表示单词是否出现在文档中，而是使用一个浮点值，该值与单词在语料库中出现的频率相关。\n",
|
||||
"\n",
|
||||
"更正式地说，单词 $i$ 在文档 $j$ 中的权重 $w_{ij}$ 定义为：\n",
|
||||
"$$\n",
|
||||
"w_{ij} = tf_{ij}\\times\\log({N\\over df_i})\n",
|
||||
"$$\n",
|
||||
"其中：\n",
|
||||
"* $tf_{ij}$ 是单词 $i$ 在文档 $j$ 中出现的次数，即我们之前看到的 BoW 值\n",
|
||||
"* $N$ 是语料库中的文档总数\n",
|
||||
"* $df_i$ 是整个语料库中包含单词 $i$ 的文档数量\n",
|
||||
"\n",
|
||||
"TF-IDF 值 $w_{ij}$ 随着单词在文档中出现的次数成比例增加，同时会根据语料库中包含该单词的文档数量进行调整。这有助于解决某些单词比其他单词更频繁出现的问题。例如，如果某个单词出现在语料库的*每一篇*文档中，那么 $df_i=N$，且 $w_{ij}=0$，这些词将被完全忽略。\n",
|
||||
"\n",
|
||||
"你可以使用 Scikit Learn 轻松创建文本的 TF-IDF 向量化：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([[0.43381609, 0. , 0.43381609, 0. , 0.65985664,\n",
|
||||
" 0.43381609, 0. , 0. , 0. , 0. ,\n",
|
||||
" 0. , 0. , 0. , 0. , 0. ,\n",
|
||||
" 0. ]])"
|
||||
]
|
||||
},
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.feature_extraction.text import TfidfVectorizer\n",
|
||||
"vectorizer = TfidfVectorizer(ngram_range=(1,2))\n",
|
||||
"vectorizer.fit_transform(corpus)\n",
|
||||
"vectorizer.transform(['My dog likes hot dogs on a hot day.']).toarray()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在 Keras 中，通过传递参数 `output_mode='tf-idf'`，`TextVectorization` 层可以自动计算 TF-IDF 频率。让我们重复上面的代码，看看使用 TF-IDF 是否能提高准确性：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training vectorizer\n",
|
||||
"938/938 [==============================] - 12s 12ms/step - loss: 0.4197 - acc: 0.8662 - val_loss: 0.3432 - val_acc: 0.8849\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<keras.callbacks.History at 0x20c729dfd30>"
|
||||
]
|
||||
},
|
||||
"execution_count": 17,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = keras.models.Sequential([\n",
|
||||
" keras.layers.experimental.preprocessing.TextVectorization(max_tokens=vocab_size,output_mode='tf-idf'),\n",
|
||||
" keras.layers.Dense(4,input_shape=(vocab_size,), activation='softmax')\n",
|
||||
"])\n",
|
||||
"print(\"Training vectorizer\")\n",
|
||||
"model.layers[0].adapt(ds_train.take(500).map(extract_text))\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['acc'])\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(batch_size),validation_data=ds_test.map(tupelize).batch(batch_size))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 结论\n",
|
||||
"\n",
|
||||
"尽管TF-IDF表示法为不同的词赋予了频率权重，但它无法表达词语的含义或顺序。正如著名语言学家J.R. Firth在1935年所说：“一个词的完整含义总是与上下文相关，任何脱离上下文的意义研究都不值得认真对待。”在课程的后续部分，我们将学习如何通过语言模型从文本中捕捉上下文信息。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。因使用本翻译而导致的任何误解或误读，我们概不负责。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "0cb620c6d4b9f7a635928804c26cf22403d89d98d79684e4529119355ee6d5a5"
|
||||
},
|
||||
"kernel_info": {
|
||||
"name": "conda-env-py37_tensorflow-py"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "py37_tensorflow",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"nteract": {
|
||||
"version": "nteract-front-end@1.0.0"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "19b43951d55b377a76209c24c1f017e4",
|
||||
"translation_date": "2025-08-31T11:00:44+00:00",
|
||||
"source_file": "lessons/5-NLP/13-TextRep/TextRepresentationTF.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# 作业：笔记本
|
||||
|
||||
使用与本课程相关的笔记本（可以选择 PyTorch 或 TensorFlow 版本），使用您自己的数据集重新运行它们，例如从 Kaggle 获取并注明出处。重写笔记本以突出您自己的发现。尝试一些可能令人惊讶的创新数据集，例如 [这个关于 UFO 目击事件的数据集](https://www.kaggle.com/datasets/NUFORC/ufo-sightings) 来自 NUFORC。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对于因使用此翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,726 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 嵌入\n",
|
||||
"\n",
|
||||
"在之前的例子中，我们处理的是长度为 `vocab_size` 的高维词袋向量，并且我们显式地将低维位置表示向量转换为稀疏的独热表示。这种独热表示并不节省内存，而且每个单词都是彼此独立处理的，也就是说，独热编码的向量无法表达单词之间的语义相似性。\n",
|
||||
"\n",
|
||||
"在本单元中，我们将继续探索 **News AG** 数据集。首先，让我们加载数据并从之前的笔记本中获取一些定义。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Loading dataset...\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"d:\\WORK\\ai-for-beginners\\5-NLP\\14-Embeddings\\data\\train.csv: 29.5MB [00:01, 18.8MB/s] \n",
|
||||
"d:\\WORK\\ai-for-beginners\\5-NLP\\14-Embeddings\\data\\test.csv: 1.86MB [00:00, 11.2MB/s] \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Building vocab...\n",
|
||||
"Vocab size = 95812\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"import torchtext\n",
|
||||
"import numpy as np\n",
|
||||
"from torchnlp import *\n",
|
||||
"train_dataset, test_dataset, classes, vocab = load_dataset()\n",
|
||||
"vocab_size = len(vocab)\n",
|
||||
"print(\"Vocab size = \",vocab_size)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 什么是嵌入？\n",
|
||||
"\n",
|
||||
"**嵌入**的概念是用低维密集向量来表示单词，这些向量能够在某种程度上反映单词的语义意义。稍后我们会讨论如何构建有意义的单词嵌入，但现在可以简单地将嵌入理解为一种降低单词向量维度的方法。\n",
|
||||
"\n",
|
||||
"嵌入层会将一个单词作为输入，并生成指定`embedding_size`的输出向量。从某种意义上说，它与`Linear`层非常相似，但嵌入层不需要接收独热编码向量，而是可以直接接收单词编号作为输入。\n",
|
||||
"\n",
|
||||
"通过将嵌入层作为网络的第一层，我们可以从词袋模型切换到**嵌入袋**模型。在嵌入袋模型中，我们首先将文本中的每个单词转换为对应的嵌入向量，然后对所有这些嵌入向量进行某种聚合操作，例如`sum`（求和）、`average`（平均）或`max`（最大值）。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"我们的分类器神经网络将从嵌入层开始，然后是聚合层，最后是线性分类器：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class EmbedClassifier(torch.nn.Module):\n",
|
||||
" def __init__(self, vocab_size, embed_dim, num_class):\n",
|
||||
" super().__init__()\n",
|
||||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||||
" self.fc = torch.nn.Linear(embed_dim, num_class)\n",
|
||||
"\n",
|
||||
" def forward(self, x):\n",
|
||||
" x = self.embedding(x)\n",
|
||||
" x = torch.mean(x,dim=1)\n",
|
||||
" return self.fc(x)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 处理变量序列长度\n",
|
||||
"\n",
|
||||
"由于这种架构，我们需要以特定方式创建网络的微批次。在前一个单元中，使用词袋模型（BoW）时，微批次中的所有 BoW 张量都具有相同的大小 `vocab_size`，无论文本序列的实际长度是多少。一旦我们转向词嵌入，每个文本样本中会包含不同数量的单词，而在将这些样本组合成微批次时，我们需要进行一些填充。\n",
|
||||
"\n",
|
||||
"这可以通过向数据源提供 `collate_fn` 函数的相同技术来实现：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def padify(b):\n",
|
||||
" # b is the list of tuples of length batch_size\n",
|
||||
" # - first element of a tuple = label, \n",
|
||||
" # - second = feature (text sequence)\n",
|
||||
" # build vectorized sequence\n",
|
||||
" v = [encode(x[1]) for x in b]\n",
|
||||
" # first, compute max length of a sequence in this minibatch\n",
|
||||
" l = max(map(len,v))\n",
|
||||
" return ( # tuple of two tensors - labels and features\n",
|
||||
" torch.LongTensor([t[0]-1 for t in b]),\n",
|
||||
" torch.stack([torch.nn.functional.pad(torch.tensor(t),(0,l-len(t)),mode='constant',value=0) for t in v])\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=padify, shuffle=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 训练嵌入分类器\n",
|
||||
"\n",
|
||||
"现在我们已经定义了合适的数据加载器，可以使用我们在上一单元中定义的训练函数来训练模型：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3200: acc=0.6415625\n",
|
||||
"6400: acc=0.6865625\n",
|
||||
"9600: acc=0.7103125\n",
|
||||
"12800: acc=0.726953125\n",
|
||||
"16000: acc=0.739375\n",
|
||||
"19200: acc=0.75046875\n",
|
||||
"22400: acc=0.7572321428571429\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(0.889799795315499, 0.7623160588611644)"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"net = EmbedClassifier(vocab_size,32,len(classes)).to(device)\n",
|
||||
"train_epoch(net,train_loader, lr=1, epoch_size=25000)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意**：我们这里只训练了25k条记录（少于一个完整的周期）以节省时间，但您可以继续训练，编写一个函数来训练多个周期，并尝试调整学习率参数以获得更高的准确性。您应该能够达到大约90%的准确率。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### EmbeddingBag层与可变长度序列表示\n",
|
||||
"\n",
|
||||
"在之前的架构中，我们需要将所有序列填充到相同的长度，以便将它们放入一个小批量中。这并不是表示可变长度序列的最有效方式——另一种方法是使用**偏移量**向量，它可以存储一个大向量中所有序列的偏移量。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"> **注意**：在上图中，我们展示的是字符序列，但在我们的示例中，我们处理的是单词序列。然而，用偏移量向量表示序列的基本原理是相同的。\n",
|
||||
"\n",
|
||||
"为了使用偏移量表示，我们使用[`EmbeddingBag`](https://pytorch.org/docs/stable/generated/torch.nn.EmbeddingBag.html)层。它与`Embedding`类似，但它以内容向量和偏移量向量作为输入，并且还包括一个平均层，可以是`mean`、`sum`或`max`。\n",
|
||||
"\n",
|
||||
"以下是使用`EmbeddingBag`的修改后的网络：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class EmbedClassifier(torch.nn.Module):\n",
|
||||
" def __init__(self, vocab_size, embed_dim, num_class):\n",
|
||||
" super().__init__()\n",
|
||||
" self.embedding = torch.nn.EmbeddingBag(vocab_size, embed_dim)\n",
|
||||
" self.fc = torch.nn.Linear(embed_dim, num_class)\n",
|
||||
"\n",
|
||||
" def forward(self, text, off):\n",
|
||||
" x = self.embedding(text, off)\n",
|
||||
" return self.fc(x)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"为了准备用于训练的数据集，我们需要提供一个转换函数来准备偏移向量：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def offsetify(b):\n",
|
||||
" # first, compute data tensor from all sequences\n",
|
||||
" x = [torch.tensor(encode(t[1])) for t in b]\n",
|
||||
" # now, compute the offsets by accumulating the tensor of sequence lengths\n",
|
||||
" o = [0] + [len(t) for t in x]\n",
|
||||
" o = torch.tensor(o[:-1]).cumsum(dim=0)\n",
|
||||
" return ( \n",
|
||||
" torch.LongTensor([t[0]-1 for t in b]), # labels\n",
|
||||
" torch.cat(x), # text \n",
|
||||
" o\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=offsetify, shuffle=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"注意，与之前的所有示例不同，我们的网络现在接受两个参数：数据向量和偏移向量，它们的大小不同。同样，我们的数据加载器也为我们提供了3个值而不是2个：文本和偏移向量都作为特征提供。因此，我们需要稍微调整我们的训练函数来处理这一点：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3200: acc=0.6153125\n",
|
||||
"6400: acc=0.6615625\n",
|
||||
"9600: acc=0.6932291666666667\n",
|
||||
"12800: acc=0.715078125\n",
|
||||
"16000: acc=0.7270625\n",
|
||||
"19200: acc=0.7382291666666667\n",
|
||||
"22400: acc=0.7486160714285715\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(22.771553103007037, 0.7551983365323096)"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"net = EmbedClassifier(vocab_size,32,len(classes)).to(device)\n",
|
||||
"\n",
|
||||
"def train_epoch_emb(net,dataloader,lr=0.01,optimizer=None,loss_fn = torch.nn.CrossEntropyLoss(),epoch_size=None, report_freq=200):\n",
|
||||
" optimizer = optimizer or torch.optim.Adam(net.parameters(),lr=lr)\n",
|
||||
" loss_fn = loss_fn.to(device)\n",
|
||||
" net.train()\n",
|
||||
" total_loss,acc,count,i = 0,0,0,0\n",
|
||||
" for labels,text,off in dataloader:\n",
|
||||
" optimizer.zero_grad()\n",
|
||||
" labels,text,off = labels.to(device), text.to(device), off.to(device)\n",
|
||||
" out = net(text, off)\n",
|
||||
" loss = loss_fn(out,labels) #cross_entropy(out,labels)\n",
|
||||
" loss.backward()\n",
|
||||
" optimizer.step()\n",
|
||||
" total_loss+=loss\n",
|
||||
" _,predicted = torch.max(out,1)\n",
|
||||
" acc+=(predicted==labels).sum()\n",
|
||||
" count+=len(labels)\n",
|
||||
" i+=1\n",
|
||||
" if i%report_freq==0:\n",
|
||||
" print(f\"{count}: acc={acc.item()/count}\")\n",
|
||||
" if epoch_size and count>epoch_size:\n",
|
||||
" break\n",
|
||||
" return total_loss.item()/count, acc.item()/count\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"train_epoch_emb(net,train_loader, lr=4, epoch_size=25000)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 语义嵌入：Word2Vec\n",
|
||||
"\n",
|
||||
"在之前的例子中，模型的嵌入层学习了将单词映射为向量表示，但这种表示并没有太多语义上的意义。如果能够学习一种向量表示，使得相似的单词或同义词在某种向量距离（例如欧几里得距离）上彼此接近，那就更好了。\n",
|
||||
"\n",
|
||||
"为了实现这一点，我们需要以特定的方式在大量文本集合上预训练嵌入模型。最早的一种训练语义嵌入的方法被称为 [Word2Vec](https://en.wikipedia.org/wiki/Word2vec)。它基于两种主要架构，用于生成单词的分布式表示：\n",
|
||||
"\n",
|
||||
" - **连续词袋模型** (CBoW) —— 在这种架构中，我们训练模型根据上下文预测一个单词。给定 ngram $(W_{-2},W_{-1},W_0,W_1,W_2)$，模型的目标是从 $(W_{-2},W_{-1},W_1,W_2)$ 中预测 $W_0$。\n",
|
||||
" - **连续跳词模型** (Skip-Gram) 与 CBoW 相反。模型使用上下文窗口中的单词来预测当前单词。\n",
|
||||
"\n",
|
||||
"CBoW 的训练速度更快，而 Skip-Gram 虽然较慢，但在表示不常见单词方面表现更好。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"为了实验基于 Google News 数据集预训练的 Word2Vec 嵌入，我们可以使用 **gensim** 库。下面我们找到与“neural”最相似的单词：\n",
|
||||
"\n",
|
||||
"> **注意:** 当你第一次创建单词向量时，下载它们可能需要一些时间！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import gensim.downloader as api\n",
|
||||
"w2v = api.load('word2vec-google-news-300')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"neuronal -> 0.7804799675941467\n",
|
||||
"neurons -> 0.7326500415802002\n",
|
||||
"neural_circuits -> 0.7252851724624634\n",
|
||||
"neuron -> 0.7174385190010071\n",
|
||||
"cortical -> 0.6941086649894714\n",
|
||||
"brain_circuitry -> 0.6923246383666992\n",
|
||||
"synaptic -> 0.6699118614196777\n",
|
||||
"neural_circuitry -> 0.6638563275337219\n",
|
||||
"neurochemical -> 0.6555314064025879\n",
|
||||
"neuronal_activity -> 0.6531826257705688\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for w,p in w2v.most_similar('neural'):\n",
|
||||
" print(f\"{w} -> {p}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们还可以从单词计算向量嵌入，用于训练分类模型（为清晰起见，我们仅显示向量的前20个分量）：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([ 0.01226807, 0.06225586, 0.10693359, 0.05810547, 0.23828125,\n",
|
||||
" 0.03686523, 0.05151367, -0.20703125, 0.01989746, 0.10058594,\n",
|
||||
" -0.03759766, -0.1015625 , -0.15820312, -0.08105469, -0.0390625 ,\n",
|
||||
" -0.05053711, 0.16015625, 0.2578125 , 0.10058594, -0.25976562],\n",
|
||||
" dtype=float32)"
|
||||
]
|
||||
},
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"w2v.word_vec('play')[:20]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"语义嵌入的一个伟大之处在于，你可以操控向量编码来改变语义。例如，我们可以要求找到一个词，其向量表示尽可能接近词语*国王*和*女人*，并尽可能远离词语*男人*：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"('queen', 0.7118192911148071)"
|
||||
]
|
||||
},
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"w2v.most_similar(positive=['king','woman'],negative=['man'])[0]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"CBoW 和 Skip-Grams 都是“预测型”嵌入，因为它们只考虑局部上下文。Word2Vec 并未利用全局上下文。\n",
|
||||
"\n",
|
||||
"**FastText** 基于 Word2Vec，通过为每个单词以及单词中的字符 n-gram 学习向量表示来扩展功能。在每次训练步骤中，这些表示的值会被平均成一个向量。虽然这增加了预训练的计算量，但它使得词嵌入能够编码子词信息。\n",
|
||||
"\n",
|
||||
"另一种方法 **GloVe** 利用了共现矩阵的概念，使用神经方法将共现矩阵分解为更具表现力和非线性的词向量。\n",
|
||||
"\n",
|
||||
"你可以通过将嵌入模型切换为 FastText 和 GloVe 来尝试示例，因为 gensim 支持多种不同的词嵌入模型。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 在 PyTorch 中使用预训练的嵌入\n",
|
||||
"\n",
|
||||
"我们可以修改上面的示例，在嵌入层的矩阵中预填充语义嵌入，例如 Word2Vec。需要注意的是，预训练嵌入的词汇表和我们的文本语料库的词汇表可能不完全匹配，因此我们会用随机值初始化缺失词的权重：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Embedding size: 300\n",
|
||||
"Populating matrix, this will take some time...Done, found 41080 words, 54732 words missing\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"embed_size = len(w2v.get_vector('hello'))\n",
|
||||
"print(f'Embedding size: {embed_size}')\n",
|
||||
"\n",
|
||||
"net = EmbedClassifier(vocab_size,embed_size,len(classes))\n",
|
||||
"\n",
|
||||
"print('Populating matrix, this will take some time...',end='')\n",
|
||||
"found, not_found = 0,0\n",
|
||||
"for i,w in enumerate(vocab.get_itos()):\n",
|
||||
" try:\n",
|
||||
" net.embedding.weight[i].data = torch.tensor(w2v.get_vector(w))\n",
|
||||
" found+=1\n",
|
||||
" except:\n",
|
||||
" net.embedding.weight[i].data = torch.normal(0.0,1.0,(embed_size,))\n",
|
||||
" not_found+=1\n",
|
||||
"\n",
|
||||
"print(f\"Done, found {found} words, {not_found} words missing\")\n",
|
||||
"net = net.to(device)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在让我们训练我们的模型。请注意，由于嵌入层大小更大，因此参数数量显著增加，训练模型所需的时间比前一个例子要长得多。此外，正因为如此，如果我们想避免过拟合，可能需要在更多的例子上训练我们的模型。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3200: acc=0.6359375\n",
|
||||
"6400: acc=0.68109375\n",
|
||||
"9600: acc=0.7067708333333333\n",
|
||||
"12800: acc=0.723671875\n",
|
||||
"16000: acc=0.73625\n",
|
||||
"19200: acc=0.7463541666666667\n",
|
||||
"22400: acc=0.7560714285714286\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(214.1013875559821, 0.7626759436980166)"
|
||||
]
|
||||
},
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"train_epoch_emb(net,train_loader, lr=4, epoch_size=25000)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在我们的案例中，准确率并没有显著提升，这可能是由于词汇差异较大。 \n",
|
||||
"为了解决词汇差异的问题，我们可以采用以下解决方案： \n",
|
||||
"* 重新训练 word2vec 模型以适应我们的词汇 \n",
|
||||
"* 使用预训练的 word2vec 模型的词汇加载我们的数据集。在加载数据集时，可以指定使用的词汇。 \n",
|
||||
"\n",
|
||||
"后一种方法似乎更简单，尤其是因为 PyTorch 的 `torchtext` 框架内置了对嵌入的支持。例如，我们可以通过以下方式实例化基于 GloVe 的词汇： \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|█████████▉| 399999/400000 [00:15<00:00, 25411.14it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vocab = torchtext.vocab.GloVe(name='6B', dim=50)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"加载的词汇表具有以下基本操作：\n",
|
||||
"* `vocab.stoi` 字典允许我们将单词转换为其在字典中的索引\n",
|
||||
"* `vocab.itos` 则执行相反的操作——将数字转换为单词\n",
|
||||
"* `vocab.vectors` 是嵌入向量的数组，因此要获取单词 `s` 的嵌入，我们需要使用 `vocab.vectors[vocab.stoi[s]]`\n",
|
||||
"\n",
|
||||
"以下是操作嵌入的一个示例，用来展示方程 **kind-man+woman = queen**（我稍微调整了一下系数以使其生效）：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"'queen'"
|
||||
]
|
||||
},
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# get the vector corresponding to kind-man+woman\n",
|
||||
"qvec = vocab.vectors[vocab.stoi['king']]-vocab.vectors[vocab.stoi['man']]+1.3*vocab.vectors[vocab.stoi['woman']]\n",
|
||||
"# find the index of the closest embedding vector \n",
|
||||
"d = torch.sum((vocab.vectors-qvec)**2,dim=1)\n",
|
||||
"min_idx = torch.argmin(d)\n",
|
||||
"# find the corresponding word\n",
|
||||
"vocab.itos[min_idx]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"要使用这些嵌入训练分类器，我们首先需要使用GloVe词汇表对数据集进行编码：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def offsetify(b):\n",
|
||||
" # first, compute data tensor from all sequences\n",
|
||||
" x = [torch.tensor(encode(t[1],voc=vocab)) for t in b] # pass the instance of vocab to encode function!\n",
|
||||
" # now, compute the offsets by accumulating the tensor of sequence lengths\n",
|
||||
" o = [0] + [len(t) for t in x]\n",
|
||||
" o = torch.tensor(o[:-1]).cumsum(dim=0)\n",
|
||||
" return ( \n",
|
||||
" torch.LongTensor([t[0]-1 for t in b]), # labels\n",
|
||||
" torch.cat(x), # text \n",
|
||||
" o\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"正如我们上面所看到的，所有向量嵌入都存储在 `vocab.vectors` 矩阵中。通过简单的复制操作，可以非常轻松地将这些权重加载到嵌入层的权重中：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"net = EmbedClassifier(len(vocab),len(vocab.vectors[0]),len(classes))\n",
|
||||
"net.embedding.weight.data = vocab.vectors\n",
|
||||
"net = net.to(device)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在让我们训练我们的模型，看看是否能获得更好的结果：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3200: acc=0.6271875\n",
|
||||
"6400: acc=0.68078125\n",
|
||||
"9600: acc=0.7030208333333333\n",
|
||||
"12800: acc=0.71984375\n",
|
||||
"16000: acc=0.7346875\n",
|
||||
"19200: acc=0.7455729166666667\n",
|
||||
"22400: acc=0.7529464285714286\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(35.53972978646833, 0.7575175943698017)"
|
||||
]
|
||||
},
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=offsetify, shuffle=True)\n",
|
||||
"train_epoch_emb(net,train_loader, lr=4, epoch_size=25000)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们没有看到准确率显著提高的原因之一是因为我们数据集中的一些词在预训练的GloVe词汇表中缺失，因此它们实际上被忽略了。为了克服这一问题，我们可以在我们的数据集上训练自己的嵌入。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 上下文嵌入\n",
|
||||
"\n",
|
||||
"传统预训练嵌入表示（如 Word2Vec）的一个主要局限是词义消歧问题。虽然预训练嵌入可以在一定程度上捕捉单词在上下文中的含义，但每个单词的所有可能含义都会被编码到同一个嵌入中。这可能会导致下游模型出现问题，因为许多单词（例如“play”）的含义会根据使用的上下文而有所不同。\n",
|
||||
"\n",
|
||||
"例如，“play”在以下两句话中的含义就完全不同：\n",
|
||||
"- 我去剧院看了一场**戏剧**。\n",
|
||||
"- 约翰想和他的朋友们一起**玩**。\n",
|
||||
"\n",
|
||||
"上述预训练嵌入将“play”的这两种含义表示为同一个嵌入。为了克服这一局限，我们需要基于**语言模型**来构建嵌入。语言模型是在大规模文本语料库上训练的，它*了解*单词如何在不同的上下文中组合。关于上下文嵌入的讨论超出了本教程的范围，但我们将在下一单元讨论语言模型时回到这个话题。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "0cb620c6d4b9f7a635928804c26cf22403d89d98d79684e4529119355ee6d5a5"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "py37_pytorch",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "f50b026abce5cf36783a560ea72cb9b1",
|
||||
"translation_date": "2025-08-31T10:55:41+00:00",
|
||||
"source_file": "lessons/5-NLP/14-Embeddings/EmbeddingsPyTorch.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,695 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 嵌入\n",
|
||||
"\n",
|
||||
"在之前的例子中，我们处理的是长度为 `vocab_size` 的高维词袋向量，并且我们将低维位置表示向量显式转换为稀疏的独热表示。这种独热表示并不节省内存。此外，每个单词都被独立对待，因此独热编码的向量无法表达单词之间的语义相似性。\n",
|
||||
"\n",
|
||||
"在本单元中，我们将继续探索 **News AG** 数据集。首先，让我们加载数据并获取上一单元中的一些定义。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorflow import keras\n",
|
||||
"import tensorflow_datasets as tfds\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"ds_train, ds_test = tfds.load('ag_news_subset').values()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 什么是嵌入？\n",
|
||||
"\n",
|
||||
"**嵌入**的核心思想是使用低维稠密向量来表示单词，这些向量能够反映单词的语义意义。稍后我们会讨论如何构建有意义的单词嵌入，但现在可以简单地将嵌入理解为一种降低单词向量维度的方法。\n",
|
||||
"\n",
|
||||
"嵌入层以单词作为输入，并输出一个指定的`embedding_size`向量。从某种意义上说，它与`Dense`层非常相似，但不同的是，嵌入层可以直接接受单词编号，而不是一个独热编码向量。\n",
|
||||
"\n",
|
||||
"通过将嵌入层作为网络的第一层，我们可以从词袋模型切换到**嵌入袋**模型。在嵌入袋模型中，我们首先将文本中的每个单词转换为对应的嵌入向量，然后对所有嵌入向量进行某种聚合操作，例如`sum`、`average`或`max`。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"我们的分类器神经网络由以下几层组成：\n",
|
||||
"\n",
|
||||
"* `TextVectorization`层：该层以字符串作为输入，并生成一个包含标记编号的张量。我们会指定一个合理的词汇表大小`vocab_size`，并忽略使用频率较低的单词。输入形状为1，输出形状为$n$，因为结果中会有$n$个标记，每个标记包含从0到`vocab_size`的编号。\n",
|
||||
"* `Embedding`层：该层接收$n$个编号，并将每个编号缩减为一个指定长度的稠密向量（在我们的例子中长度为100）。因此，形状为$n$的输入张量将被转换为一个$n\\times 100$的张量。\n",
|
||||
"* 聚合层：该层沿着第一个轴对张量进行平均计算，即计算所有$n$个输入张量（对应不同单词）的平均值。为了实现这一层，我们会使用一个`Lambda`层，并传入计算平均值的函数。输出形状为100，它将是整个输入序列的数值表示。\n",
|
||||
"* 最后的`Dense`线性分类器。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"sequential\"\n",
|
||||
"_________________________________________________________________\n",
|
||||
" Layer (type) Output Shape Param # \n",
|
||||
"=================================================================\n",
|
||||
" text_vectorization (TextVec (None, None) 0 \n",
|
||||
" torization) \n",
|
||||
" \n",
|
||||
" embedding (Embedding) (None, None, 100) 3000000 \n",
|
||||
" \n",
|
||||
" lambda (Lambda) (None, 100) 0 \n",
|
||||
" \n",
|
||||
" dense (Dense) (None, 4) 404 \n",
|
||||
" \n",
|
||||
"=================================================================\n",
|
||||
"Total params: 3,000,404\n",
|
||||
"Trainable params: 3,000,404\n",
|
||||
"Non-trainable params: 0\n",
|
||||
"_________________________________________________________________\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vocab_size = 30000\n",
|
||||
"batch_size = 128\n",
|
||||
"\n",
|
||||
"vectorizer = keras.layers.experimental.preprocessing.TextVectorization(max_tokens=vocab_size,input_shape=(1,))\n",
|
||||
"\n",
|
||||
"model = keras.models.Sequential([\n",
|
||||
" vectorizer, \n",
|
||||
" keras.layers.Embedding(vocab_size,100),\n",
|
||||
" keras.layers.Lambda(lambda x: tf.reduce_mean(x,axis=1)),\n",
|
||||
" keras.layers.Dense(4, activation='softmax')\n",
|
||||
"])\n",
|
||||
"model.summary()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在 `summary` 输出中，**output shape** 列的第一个张量维度 `None` 表示小批量的大小，第二个维度表示标记序列的长度。小批量中的所有标记序列长度都不同。我们将在下一节讨论如何处理这个问题。\n",
|
||||
"\n",
|
||||
"现在让我们开始训练网络：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training vectorizer\n",
|
||||
"938/938 [==============================] - 20s 20ms/step - loss: 0.7891 - acc: 0.8155 - val_loss: 0.4470 - val_acc: 0.8642\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<keras.callbacks.History at 0x22255515100>"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def extract_text(x):\n",
|
||||
" return x['title']+' '+x['description']\n",
|
||||
"\n",
|
||||
"def tupelize(x):\n",
|
||||
" return (extract_text(x),x['label'])\n",
|
||||
"\n",
|
||||
"print(\"Training vectorizer\")\n",
|
||||
"vectorizer.adapt(ds_train.take(500).map(extract_text))\n",
|
||||
"\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'])\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(batch_size),validation_data=ds_test.map(tupelize).batch(batch_size))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"nteract": {
|
||||
"transient": {
|
||||
"deleting": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"> **注意** 我们正在基于数据的一个子集构建向量化器。这是为了加快处理速度，但可能会导致我们的文本中并非所有的标记都出现在词汇表中。在这种情况下，这些标记将被忽略，这可能会导致准确性略有下降。然而，在实际情况下，文本的一个子集通常可以提供一个良好的词汇估计。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 处理变量序列长度\n",
|
||||
"\n",
|
||||
"让我们来了解一下小批量训练的过程。在上面的例子中，输入张量的维度是1，我们使用长度为128的小批量，因此张量的实际大小是 $128 \\times 1$。然而，每个句子中的标记数量是不同的。如果我们对单个输入应用 `TextVectorization` 层，返回的标记数量会有所不同，这取决于文本是如何被分词的：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"tf.Tensor([ 1 45], shape=(2,), dtype=int64)\n",
|
||||
"tf.Tensor([ 112 1271 1 3 1747 158], shape=(6,), dtype=int64)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(vectorizer('Hello, world!'))\n",
|
||||
"print(vectorizer('I am glad to meet you!'))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"然而，当我们将向量化器应用于多个序列时，它必须生成一个矩形形状的张量，因此会用PAD标记（在我们的例子中是零）填充未使用的元素：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tf.Tensor: shape=(2, 6), dtype=int64, numpy=\n",
|
||||
"array([[ 1, 45, 0, 0, 0, 0],\n",
|
||||
" [ 112, 1271, 1, 3, 1747, 158]], dtype=int64)>"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vectorizer(['Hello, world!','I am glad to meet you!'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在这里我们可以看到嵌入：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([[[ 1.53059261e-02, 6.80514947e-02, 3.14026810e-02, ...,\n",
|
||||
" -8.92002955e-02, 1.52911525e-04, -5.65562584e-02],\n",
|
||||
" [ 2.57456154e-01, 2.79364467e-01, -2.03605562e-01, ...,\n",
|
||||
" -2.07474351e-01, 8.31158683e-02, -2.03911960e-01],\n",
|
||||
" [ 3.98201384e-02, -8.03454965e-03, 2.39790026e-02, ...,\n",
|
||||
" -7.18549127e-04, 2.66963355e-02, -4.30646613e-02],\n",
|
||||
" [ 3.98201384e-02, -8.03454965e-03, 2.39790026e-02, ...,\n",
|
||||
" -7.18549127e-04, 2.66963355e-02, -4.30646613e-02],\n",
|
||||
" [ 3.98201384e-02, -8.03454965e-03, 2.39790026e-02, ...,\n",
|
||||
" -7.18549127e-04, 2.66963355e-02, -4.30646613e-02],\n",
|
||||
" [ 3.98201384e-02, -8.03454965e-03, 2.39790026e-02, ...,\n",
|
||||
" -7.18549127e-04, 2.66963355e-02, -4.30646613e-02]],\n",
|
||||
"\n",
|
||||
" [[ 1.89674050e-01, 2.61548996e-01, -3.67433839e-02, ...,\n",
|
||||
" -2.07366899e-01, -1.05442435e-01, -2.36952081e-01],\n",
|
||||
" [ 6.16133213e-02, 1.80511594e-01, 9.77298319e-02, ...,\n",
|
||||
" -5.46628237e-02, -1.07340455e-01, -1.06589928e-01],\n",
|
||||
" [ 1.53059261e-02, 6.80514947e-02, 3.14026810e-02, ...,\n",
|
||||
" -8.92002955e-02, 1.52911525e-04, -5.65562584e-02],\n",
|
||||
" [-4.84890305e-02, -8.41715634e-02, 1.51529670e-01, ...,\n",
|
||||
" 1.28192469e-01, -7.77286515e-02, 1.26041949e-01],\n",
|
||||
" [-4.17212099e-02, -5.60694858e-02, 4.08860669e-02, ...,\n",
|
||||
" 8.70475471e-02, 8.92383084e-02, 1.67974353e-01],\n",
|
||||
" [ 2.85779923e-01, 4.57767487e-01, 4.52292450e-02, ...,\n",
|
||||
" -1.97419018e-01, -2.04659685e-01, -2.79758364e-01]]],\n",
|
||||
" dtype=float32)"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.layers[1](vectorizer(['Hello, world!','I am glad to meet you!'])).numpy()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意**：为了尽量减少填充量，在某些情况下，将数据集中所有序列按长度递增顺序（或更准确地说，按标记数量）排序是有意义的。这将确保每个小批量包含长度相似的序列。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 语义嵌入：Word2Vec\n",
|
||||
"\n",
|
||||
"在之前的例子中，嵌入层学习了将单词映射到向量表示的方法，但这些表示并没有语义意义。如果能学习一种向量表示，使得相似的单词或同义词在某种向量距离（例如欧几里得距离）上彼此接近，那就更好了。\n",
|
||||
"\n",
|
||||
"为此，我们需要使用像 [Word2Vec](https://en.wikipedia.org/wiki/Word2vec) 这样的技术，在大量文本集合上预训练嵌入模型。它基于两种主要架构，用于生成单词的分布式表示：\n",
|
||||
"\n",
|
||||
" - **连续词袋模型** (CBoW)，我们训练模型通过上下文预测一个单词。给定 ngram $(W_{-2},W_{-1},W_0,W_1,W_2)$，模型的目标是通过 $(W_{-2},W_{-1},W_1,W_2)$ 预测 $W_0$。\n",
|
||||
" - **连续跳词模型** (Skip-Gram) 与 CBoW 相反。模型使用上下文窗口中的单词来预测当前单词。\n",
|
||||
"\n",
|
||||
"CBoW 的速度更快，而 Skip-Gram 虽然较慢，但在表示不常见单词方面表现更好。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"为了试验在 Google News 数据集上预训练的 Word2Vec 嵌入，我们可以使用 **gensim** 库。下面我们找到与“neural”最相似的单词。\n",
|
||||
"\n",
|
||||
"> **注意:** 当你首次创建单词向量时，下载可能需要一些时间！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import gensim.downloader as api\n",
|
||||
"w2v = api.load('word2vec-google-news-300')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"neuronal -> 0.7804799675941467\n",
|
||||
"neurons -> 0.7326500415802002\n",
|
||||
"neural_circuits -> 0.7252851724624634\n",
|
||||
"neuron -> 0.7174385190010071\n",
|
||||
"cortical -> 0.6941086649894714\n",
|
||||
"brain_circuitry -> 0.6923246383666992\n",
|
||||
"synaptic -> 0.6699118614196777\n",
|
||||
"neural_circuitry -> 0.6638563275337219\n",
|
||||
"neurochemical -> 0.6555314064025879\n",
|
||||
"neuronal_activity -> 0.6531826257705688\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for w,p in w2v.most_similar('neural'):\n",
|
||||
" print(f\"{w} -> {p}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们还可以从单词中提取向量嵌入，用于训练分类模型。嵌入有300个组件，但这里为了清晰起见，我们只显示向量的前20个组件：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([ 0.01226807, 0.06225586, 0.10693359, 0.05810547, 0.23828125,\n",
|
||||
" 0.03686523, 0.05151367, -0.20703125, 0.01989746, 0.10058594,\n",
|
||||
" -0.03759766, -0.1015625 , -0.15820312, -0.08105469, -0.0390625 ,\n",
|
||||
" -0.05053711, 0.16015625, 0.2578125 , 0.10058594, -0.25976562],\n",
|
||||
" dtype=float32)"
|
||||
]
|
||||
},
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"w2v['play'][:20]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"语义嵌入的伟大之处在于您可以根据语义操纵向量编码。例如，我们可以要求找到一个单词，其向量表示尽可能接近*国王*和*女人*，并尽可能远离*男人*：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"('queen', 0.7118192911148071)"
|
||||
]
|
||||
},
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"w2v.most_similar(positive=['king','woman'],negative=['man'])[0]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"source": [
|
||||
"上面的例子使用了一些内部的GenSym魔法，但其底层逻辑实际上非常简单。关于嵌入的一个有趣之处在于，你可以对嵌入向量执行正常的向量操作，这将反映出对单词**意义**的操作。上面的例子可以用向量操作来表达：我们计算出与**KING-MAN+WOMAN**对应的向量（对相应单词的向量表示执行`+`和`-`操作），然后找到字典中与该向量最接近的单词：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"'queen'"
|
||||
]
|
||||
},
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# get the vector corresponding to kind-man+woman\n",
|
||||
"qvec = w2v['king']-1.7*w2v['man']+1.7*w2v['woman']\n",
|
||||
"# find the index of the closest embedding vector \n",
|
||||
"d = np.sum((w2v.vectors-qvec)**2,axis=1)\n",
|
||||
"min_idx = np.argmin(d)\n",
|
||||
"# find the corresponding word\n",
|
||||
"w2v.index_to_key[min_idx]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **NOTE**: 我们需要在 *man* 和 *woman* 向量上添加一个小系数——试着去掉它们，看看会发生什么。\n",
|
||||
"\n",
|
||||
"为了找到最接近的向量，我们使用 TensorFlow 的工具来计算我们的向量与词汇表中所有向量之间的距离向量，然后使用 `argmin` 找到最小值对应的单词索引。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"虽然 Word2Vec 是一种表达词语语义的好方法，但它存在许多缺点，包括以下几点：\n",
|
||||
"\n",
|
||||
"* CBoW 和 skip-gram 模型都是**预测型嵌入**，它们仅考虑局部上下文。Word2Vec 没有利用全局上下文。\n",
|
||||
"* Word2Vec 没有考虑词语的**形态学**，即词语的意义可能取决于词的不同部分，例如词根。\n",
|
||||
"\n",
|
||||
"**FastText** 试图克服第二个限制，并在 Word2Vec 的基础上改进，通过学习每个词的向量表示以及词内的字符 n-gram。然后在每次训练步骤中，将这些表示的值平均成一个向量。虽然这增加了预训练的计算量，但它使词嵌入能够编码子词信息。\n",
|
||||
"\n",
|
||||
"另一种方法，**GloVe**，采用了一种不同的词嵌入方法，基于词-上下文矩阵的分解。首先，它构建了一个大型矩阵，记录词语在不同上下文中的出现次数，然后尝试以降低维度的方式表示该矩阵，同时最小化重构损失。\n",
|
||||
"\n",
|
||||
"gensim 库支持这些词嵌入方法，你可以通过修改上面的模型加载代码来尝试这些方法。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 在 Keras 中使用预训练的嵌入\n",
|
||||
"\n",
|
||||
"我们可以修改上面的示例，在嵌入层的矩阵中预填充语义嵌入，例如 Word2Vec。预训练嵌入的词汇表和文本语料库的词汇表可能不匹配，因此我们需要选择一个。在这里，我们探讨两种可能的选项：使用分词器的词汇表，或者使用 Word2Vec 嵌入的词汇表。\n",
|
||||
"\n",
|
||||
"### 使用分词器词汇表\n",
|
||||
"\n",
|
||||
"当使用分词器的词汇表时，词汇表中的一些单词会有对应的 Word2Vec 嵌入，而另一些则没有。假设我们的词汇表大小为 `vocab_size`，Word2Vec 嵌入向量的长度为 `embed_size`，那么嵌入层将由一个形状为 `vocab_size`$\\times$`embed_size` 的权重矩阵表示。我们将通过遍历词汇表来填充这个矩阵：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Embedding size: 300\n",
|
||||
"Populating matrix, this will take some time...Done, found 4551 words, 784 words missing\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"embed_size = len(w2v.get_vector('hello'))\n",
|
||||
"print(f'Embedding size: {embed_size}')\n",
|
||||
"\n",
|
||||
"vocab = vectorizer.get_vocabulary()\n",
|
||||
"W = np.zeros((vocab_size,embed_size))\n",
|
||||
"print('Populating matrix, this will take some time...',end='')\n",
|
||||
"found, not_found = 0,0\n",
|
||||
"for i,w in enumerate(vocab):\n",
|
||||
" try:\n",
|
||||
" W[i] = w2v.get_vector(w)\n",
|
||||
" found+=1\n",
|
||||
" except:\n",
|
||||
" # W[i] = np.random.normal(0.0,0.3,size=(embed_size,))\n",
|
||||
" not_found+=1\n",
|
||||
"\n",
|
||||
"print(f\"Done, found {found} words, {not_found} words missing\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"对于不在 Word2Vec 词汇表中的单词，我们可以选择将它们保留为零向量，或者生成一个随机向量。\n",
|
||||
"\n",
|
||||
"现在我们可以定义一个带有预训练权重的嵌入层：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"emb = keras.layers.Embedding(vocab_size,embed_size,weights=[W],trainable=False)\n",
|
||||
"model = keras.models.Sequential([\n",
|
||||
" vectorizer, emb,\n",
|
||||
" keras.layers.Lambda(lambda x: tf.reduce_mean(x,axis=1)),\n",
|
||||
" keras.layers.Dense(4, activation='softmax')\n",
|
||||
"])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"938/938 [==============================] - 10s 10ms/step - loss: 1.1075 - acc: 0.7822 - val_loss: 0.9134 - val_acc: 0.8175\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<keras.callbacks.History at 0x2220226ef10>"
|
||||
]
|
||||
},
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'])\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(batch_size),\n",
|
||||
" validation_data=ds_test.map(tupelize).batch(batch_size))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意**: 请注意，在创建 `Embedding` 时我们设置了 `trainable=False`，这意味着我们不会重新训练 Embedding 层。这可能会导致准确率略低，但可以加快训练速度。\n",
|
||||
"\n",
|
||||
"### 使用嵌入词汇表\n",
|
||||
"\n",
|
||||
"之前方法的一个问题是，TextVectorization 和 Embedding 使用的词汇表不同。为了解决这个问题，我们可以采用以下解决方案之一：\n",
|
||||
"* 使用我们的词汇表重新训练 Word2Vec 模型。\n",
|
||||
"* 使用预训练的 Word2Vec 模型中的词汇表加载我们的数据集。在加载时可以指定用于加载数据集的词汇表。\n",
|
||||
"\n",
|
||||
"后者看起来更简单，所以我们来实现它。首先，我们将创建一个 `TextVectorization` 层，并使用从 Word2Vec 嵌入中提取的指定词汇表：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"vocab = list(w2v.vocab.keys())\n",
|
||||
"vectorizer = keras.layers.experimental.preprocessing.TextVectorization(input_shape=(1,))\n",
|
||||
"vectorizer.set_vocabulary(vocab)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"gensim词嵌入库包含一个方便的函数`get_keras_embeddings`，它会自动为您创建相应的Keras嵌入层。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Epoch 1/5\n",
|
||||
"938/938 [==============================] - 20s 14ms/step - loss: 1.3377 - acc: 0.4978 - val_loss: 1.2995 - val_acc: 0.5647\n",
|
||||
"Epoch 2/5\n",
|
||||
"938/938 [==============================] - 10s 10ms/step - loss: 1.2587 - acc: 0.5722 - val_loss: 1.2339 - val_acc: 0.5842\n",
|
||||
"Epoch 3/5\n",
|
||||
"938/938 [==============================] - 10s 10ms/step - loss: 1.1980 - acc: 0.5884 - val_loss: 1.1826 - val_acc: 0.5954\n",
|
||||
"Epoch 4/5\n",
|
||||
"938/938 [==============================] - 12s 13ms/step - loss: 1.1503 - acc: 0.6002 - val_loss: 1.1417 - val_acc: 0.6018\n",
|
||||
"Epoch 5/5\n",
|
||||
"938/938 [==============================] - 11s 12ms/step - loss: 1.1120 - acc: 0.6097 - val_loss: 1.1083 - val_acc: 0.6104\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<keras.callbacks.History at 0x2220ccb81c0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = keras.models.Sequential([\n",
|
||||
" vectorizer, \n",
|
||||
" w2v.get_keras_embedding(train_embeddings=False),\n",
|
||||
" keras.layers.Lambda(lambda x: tf.reduce_mean(x,axis=1)),\n",
|
||||
" keras.layers.Dense(4, activation='softmax')\n",
|
||||
"])\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'])\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(128),validation_data=ds_test.map(tupelize).batch(128),epochs=5)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们未能看到更高准确率的原因之一是，因为我们数据集中的一些词在预训练的GloVe词汇表中缺失，因此它们实际上被忽略了。为了解决这个问题，我们可以基于我们的数据集训练自己的嵌入。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 上下文嵌入\n",
|
||||
"\n",
|
||||
"传统预训练嵌入表示（如 Word2Vec）的一个主要局限是，尽管它们可以捕捉到一个词的一些含义，但无法区分不同的含义。这可能会在下游模型中引发问题。\n",
|
||||
"\n",
|
||||
"例如，单词“play”在以下两句话中有不同的含义：\n",
|
||||
"- 我去剧院看了一场**戏剧**。\n",
|
||||
"- 约翰想和他的朋友们一起**玩**。\n",
|
||||
"\n",
|
||||
"我们提到的预训练嵌入会用相同的嵌入表示“play”这个词的两种含义。为了克服这一局限，我们需要基于**语言模型**来构建嵌入。语言模型是在大规模文本语料库上训练的，它*知道*单词在不同上下文中如何组合。讨论上下文嵌入超出了本教程的范围，但我们将在下一单元讨论语言模型时回到这个话题。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。原始语言的文档应被视为权威来源。对于重要信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "0cb620c6d4b9f7a635928804c26cf22403d89d98d79684e4529119355ee6d5a5"
|
||||
},
|
||||
"kernel_info": {
|
||||
"name": "conda-env-py37_tensorflow-py"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "py37_tensorflow",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"nteract": {
|
||||
"version": "nteract-front-end@1.0.0"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "b859482be7f61d1eadc2c6a2720a37e4",
|
||||
"translation_date": "2025-08-31T10:52:41+00:00",
|
||||
"source_file": "lessons/5-NLP/14-Embeddings/EmbeddingsTF.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
# 嵌入
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/27)
|
||||
|
||||
在基于 BoW 或 TF/IDF 训练分类器时，我们使用的是长度为 `vocab_size` 的高维词袋向量，并且我们显式地将低维位置表示向量转换为稀疏的独热表示。然而，这种独热表示并不节省内存。此外，每个单词都被独立对待，即独热编码的向量无法表达单词之间的语义相似性。
|
||||
|
||||
**嵌入**的想法是用低维密集向量来表示单词，这些向量能够以某种方式反映单词的语义含义。稍后我们会讨论如何构建有意义的词嵌入，但现在我们只需将嵌入理解为一种降低单词向量维度的方法。
|
||||
|
||||
嵌入层会将一个单词作为输入，并生成指定 `embedding_size` 的输出向量。从某种意义上说，它与 `Linear` 层非常相似，但它不需要独热编码向量，而是可以直接接受单词编号作为输入，从而避免创建大型独热编码向量。
|
||||
|
||||
通过在分类器网络中使用嵌入层作为第一层，我们可以从词袋模型切换到 **嵌入袋** 模型。在嵌入袋模型中，我们首先将文本中的每个单词转换为对应的嵌入，然后对所有这些嵌入计算某种聚合函数，例如 `sum`、`average` 或 `max`。
|
||||
|
||||
|
||||
|
||||
> 图片由作者提供
|
||||
|
||||
## ✍️ 练习：嵌入
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
* [使用 PyTorch 的嵌入](EmbeddingsPyTorch.ipynb)
|
||||
* [使用 TensorFlow 的嵌入](EmbeddingsTF.ipynb)
|
||||
|
||||
## 语义嵌入：Word2Vec
|
||||
|
||||
虽然嵌入层可以学习将单词映射到向量表示，但这种表示不一定具有很强的语义意义。如果我们能学习一种向量表示，使得相似单词或同义词在某种向量距离（例如欧几里得距离）上彼此接近，那就更好了。
|
||||
|
||||
为此，我们需要以特定方式在大量文本上预训练嵌入模型。一种训练语义嵌入的方法叫做 [Word2Vec](https://en.wikipedia.org/wiki/Word2vec)。它基于两种主要架构来生成单词的分布式表示：
|
||||
|
||||
- **连续词袋** (CBoW) —— 在这种架构中，我们训练模型根据上下文预测单词。给定 ngram $(W_{-2},W_{-1},W_0,W_1,W_2)$，模型的目标是根据 $(W_{-2},W_{-1},W_1,W_2)$ 预测 $W_0$。
|
||||
- **连续跳词模型** (Skip-Gram) —— 与 CBoW 相反，模型使用上下文窗口中的单词来预测当前单词。
|
||||
|
||||
CBoW 速度更快，而 Skip-Gram 虽然较慢，但在表示不常见单词方面表现更好。
|
||||
|
||||
|
||||
|
||||
> 图片来源于 [这篇论文](https://arxiv.org/pdf/1301.3781.pdf)
|
||||
|
||||
Word2Vec 预训练嵌入（以及其他类似模型，例如 GloVe）也可以替代神经网络中的嵌入层使用。然而，我们需要处理词汇表问题，因为用于预训练 Word2Vec/GloVe 的词汇表可能与我们文本语料库中的词汇表不同。查看上述笔记本，了解如何解决这个问题。
|
||||
|
||||
## 上下文嵌入
|
||||
|
||||
传统的预训练嵌入表示（如 Word2Vec）的一个主要限制是词义消歧问题。虽然预训练嵌入可以捕捉单词在上下文中的部分含义，但每个单词的所有可能含义都被编码到同一个嵌入中。这可能会在下游模型中引发问题，因为许多单词（例如“play”）的含义会根据使用的上下文而有所不同。
|
||||
|
||||
例如，“play”在以下两个句子中的含义完全不同：
|
||||
|
||||
- 我去剧院看了一场**戏剧**。
|
||||
- 约翰想和朋友们**玩耍**。
|
||||
|
||||
上述预训练嵌入将“play”的这两种含义表示为同一个嵌入。为了克服这一限制，我们需要基于**语言模型**构建嵌入，该模型在大量文本语料库上进行训练，并且*了解*单词在不同上下文中的组合方式。讨论上下文嵌入超出了本教程的范围，但我们将在课程后续讨论语言模型时回到这个话题。
|
||||
|
||||
## 总结
|
||||
|
||||
在本课中，你学习了如何在 TensorFlow 和 PyTorch 中构建和使用嵌入层，以更好地反映单词的语义含义。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
Word2Vec 已被用于一些有趣的应用，包括生成歌词和诗歌。看看[这篇文章](https://www.politetype.com/blog/word2vec-color-poems)，作者在其中介绍了如何使用 Word2Vec 生成诗歌。还可以观看[Dan Shiffmann 的这个视频](https://www.youtube.com/watch?v=LSS_bos_TPI&ab_channel=TheCodingTrain)，了解该技术的另一种解释。然后尝试将这些技术应用到你自己的文本语料库中，可以从 Kaggle 获取数据。
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/28)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
阅读这篇关于 Word2Vec 的论文：[Efficient Estimation of Word Representations in Vector Space](https://arxiv.org/pdf/1301.3781.pdf)
|
||||
|
||||
## [作业：笔记本](assignment.md)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# 作业：笔记本
|
||||
|
||||
使用与本课程相关的笔记本（可以是 PyTorch 或 TensorFlow 版本），使用您自己的数据集重新运行它们，例如来自 Kaggle 的数据集，并注明出处。重写笔记本以突出您自己的发现。尝试使用不同类型的数据集并记录您的发现，例如使用[这些 Beatles 歌词](https://www.kaggle.com/datasets/jenlooper/beatles-lyrics)中的文本。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。因使用本翻译而引起的任何误解或误读，我们概不负责。
|
||||
|
|
@ -0,0 +1,576 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "NXTSugt6ieXh"
|
||||
},
|
||||
"source": [
|
||||
"## 训练 CBoW 模型\n",
|
||||
"\n",
|
||||
"本笔记本是 [AI for Beginners Curriculum](http://aka.ms/ai-beginners) 的一部分\n",
|
||||
"\n",
|
||||
"在这个例子中，我们将学习如何训练 CBoW 语言模型，以获得我们自己的 Word2Vec 嵌入空间。我们将使用 AG News 数据集作为文本来源。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"import torchtext\n",
|
||||
"import os\n",
|
||||
"import collections\n",
|
||||
"import builtins\n",
|
||||
"import random\n",
|
||||
"import numpy as np"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "q-UiiJUKaxHj"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "TFbR8CZaTZ1q"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"首先，让我们加载数据集并定义分词器和词汇表。我们将 `vocab_size` 设置为 5000 以稍微限制计算量。\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "HIwC7lI5T-ov"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"def load_dataset(ngrams = 1, min_freq = 1, vocab_size = 5000 , lines_cnt = 500):\n",
|
||||
" tokenizer = torchtext.data.utils.get_tokenizer('basic_english')\n",
|
||||
" print(\"Loading dataset...\")\n",
|
||||
" test_dataset, train_dataset = torchtext.datasets.AG_NEWS(root='./data')\n",
|
||||
" train_dataset = list(train_dataset)\n",
|
||||
" test_dataset = list(test_dataset)\n",
|
||||
" classes = ['World', 'Sports', 'Business', 'Sci/Tech']\n",
|
||||
" print('Building vocab...')\n",
|
||||
" counter = collections.Counter()\n",
|
||||
" for i, (_, line) in enumerate(train_dataset):\n",
|
||||
" counter.update(torchtext.data.utils.ngrams_iterator(tokenizer(line),ngrams=ngrams))\n",
|
||||
" if i == lines_cnt:\n",
|
||||
" break\n",
|
||||
" vocab = torchtext.vocab.Vocab(collections.Counter(dict(counter.most_common(vocab_size))), min_freq=min_freq)\n",
|
||||
" return train_dataset, test_dataset, classes, vocab, tokenizer"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "wdZuygtgiuLG"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"train_dataset, test_dataset, _, vocab, tokenizer = load_dataset()"
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "4d1nU1gsivGu",
|
||||
"outputId": "949fe272-ae0e-49f5-c373-6703458b3a74"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Loading dataset...\n",
|
||||
"Building vocab...\n"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"def encode(x, vocabulary, tokenizer = tokenizer):\n",
|
||||
" return [vocabulary[s] for s in tokenizer(x)]"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "1XDYNhG8ToFV"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "LIlQk6_PaHVY"
|
||||
},
|
||||
"source": [
|
||||
"## CBoW 模型\n",
|
||||
"\n",
|
||||
"CBoW 通过 $2N$ 个邻近词来预测一个单词。例如，当 $N=1$ 时，我们可以从句子 *I like to train networks* 中得到以下配对：(like,I)、(I, like)、(to, like)、(like,to)、(train,to)、(to, train)、(networks, train)、(train,networks)。这里，第一个单词是作为输入的邻近词，第二个单词是我们要预测的目标词。\n",
|
||||
"\n",
|
||||
"为了构建一个预测下一个单词的网络，我们需要将邻近词作为输入，并输出单词编号。CBoW 网络的架构如下：\n",
|
||||
"\n",
|
||||
"* 输入单词会通过嵌入层处理。这个嵌入层就是我们的 Word2Vec 嵌入，因此我们会将其单独定义为 `embedder` 变量。在这个例子中，我们将使用嵌入维度为 30，尽管你可能希望尝试更高的维度（真实的 Word2Vec 通常使用 300）。\n",
|
||||
"* 嵌入向量接着会传递到一个线性层，该层用于预测输出单词。因此它包含 `vocab_size` 个神经元。\n",
|
||||
"\n",
|
||||
"对于输出，如果我们使用 `CrossEntropyLoss` 作为损失函数，那么我们只需要提供单词编号作为期望结果，而不需要进行独热编码。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"vocab_size = len(vocab)\n",
|
||||
"\n",
|
||||
"embedder = torch.nn.Embedding(num_embeddings = vocab_size, embedding_dim = 30)\n",
|
||||
"model = torch.nn.Sequential(\n",
|
||||
" embedder,\n",
|
||||
" torch.nn.Linear(in_features = 30, out_features = vocab_size),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(model)"
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "akKTcKQKkfl2",
|
||||
"outputId": "da687e3e-a8ec-4c1a-e456-ab8cd6ac7dad"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Sequential(\n",
|
||||
" (0): Embedding(5002, 30)\n",
|
||||
" (1): Linear(in_features=30, out_features=5002, bias=True)\n",
|
||||
")\n"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Nud6jgGPaHVa"
|
||||
},
|
||||
"source": [
|
||||
"## 准备训练数据\n",
|
||||
"\n",
|
||||
"现在让我们编写一个主函数，用于从文本中计算CBoW词对。这个函数将允许我们指定窗口大小，并返回一组词对——输入词和输出词。请注意，这个函数既可以用于单词，也可以用于向量/张量——这将使我们能够在传递给`to_cbow`函数之前对文本进行编码。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "x-dsXygOieXn",
|
||||
"outputId": "c2218280-e540-40ba-9546-efe48d0d714f"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"[['like', 'I'], ['to', 'I'], ['I', 'like'], ['to', 'like'], ['train', 'like'], ['I', 'to'], ['like', 'to'], ['train', 'to'], ['networks', 'to'], ['like', 'train'], ['to', 'train'], ['networks', 'train'], ['to', 'networks'], ['train', 'networks']]\n",
|
||||
"[[232, 172], [5, 172], [172, 232], [5, 232], [0, 232], [172, 5], [232, 5], [0, 5], [1202, 5], [232, 0], [5, 0], [1202, 0], [5, 1202], [0, 1202]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def to_cbow(sent,window_size=2):\n",
|
||||
" res = []\n",
|
||||
" for i,x in enumerate(sent):\n",
|
||||
" for j in range(max(0,i-window_size),min(i+window_size+1,len(sent))):\n",
|
||||
" if i!=j:\n",
|
||||
" res.append([sent[j],x])\n",
|
||||
" return res\n",
|
||||
"\n",
|
||||
"print(to_cbow(['I','like','to','train','networks']))\n",
|
||||
"print(to_cbow(encode('I like to train networks', vocab)))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "XVaaDLjaaHVb"
|
||||
},
|
||||
"source": [
|
||||
"让我们准备训练数据集。我们将遍历所有新闻，调用`to_cbow`获取单词对列表，并将这些对添加到`X`和`Y`中。为了节省时间，我们只考虑前1万条新闻——如果你有更多时间等待，并希望获得更好的嵌入，可以轻松去掉这个限制 :)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "54b-Gd9TieXo"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"X = []\n",
|
||||
"Y = []\n",
|
||||
"for i, x in zip(range(10000), train_dataset):\n",
|
||||
" for w1, w2 in to_cbow(encode(x[1], vocab), window_size = 5):\n",
|
||||
" X.append(w1)\n",
|
||||
" Y.append(w2)\n",
|
||||
"\n",
|
||||
"X = torch.tensor(X)\n",
|
||||
"Y = torch.tensor(Y)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"我们还将把这些数据转换为一个数据集，并创建数据加载器：\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "cwWy0PzXWhN5"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"class SimpleIterableDataset(torch.utils.data.IterableDataset):\n",
|
||||
" def __init__(self, X, Y):\n",
|
||||
" super(SimpleIterableDataset).__init__()\n",
|
||||
" self.data = []\n",
|
||||
" for i in range(len(X)):\n",
|
||||
" self.data.append( (Y[i], X[i]) )\n",
|
||||
" random.shuffle(self.data)\n",
|
||||
"\n",
|
||||
" def __iter__(self):\n",
|
||||
" return iter(self.data)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "mfoAcGPFZU8p"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "e4NQ_-5waHVc"
|
||||
},
|
||||
"source": [
|
||||
"我们还将把这些数据转换为一个数据集，并创建数据加载器：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "AbLUcojlieXo"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"ds = SimpleIterableDataset(X, Y)\n",
|
||||
"dl = torch.utils.data.DataLoader(ds, batch_size = 256)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "pKQr7sXeaHVc"
|
||||
},
|
||||
"source": [
|
||||
"现在让我们进行实际训练。我们将使用`SGD`优化器，并设置较高的学习率。您也可以尝试使用其他优化器，例如`Adam`。我们将从训练10个周期开始——如果您希望获得更低的损失，可以重新运行此单元格。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"def train_epoch(net, dataloader, lr = 0.01, optimizer = None, loss_fn = torch.nn.CrossEntropyLoss(), epochs = None, report_freq = 1):\n",
|
||||
" optimizer = optimizer or torch.optim.Adam(net.parameters(), lr = lr)\n",
|
||||
" loss_fn = loss_fn.to(device)\n",
|
||||
" net.train()\n",
|
||||
"\n",
|
||||
" for i in range(epochs):\n",
|
||||
" total_loss, j = 0, 0, \n",
|
||||
" for labels, features in dataloader:\n",
|
||||
" optimizer.zero_grad()\n",
|
||||
" features, labels = features.to(device), labels.to(device)\n",
|
||||
" out = net(features)\n",
|
||||
" loss = loss_fn(out, labels)\n",
|
||||
" loss.backward()\n",
|
||||
" optimizer.step()\n",
|
||||
" total_loss += loss\n",
|
||||
" j += 1\n",
|
||||
" if i % report_freq == 0:\n",
|
||||
" print(f\"Epoch: {i+1}: loss={total_loss.item()/j}\")\n",
|
||||
"\n",
|
||||
" return total_loss.item()/j"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "HeeCYKr_KF1w"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"train_epoch(net = model, dataloader = dl, optimizer = torch.optim.SGD(model.parameters(), lr = 0.1), loss_fn = torch.nn.CrossEntropyLoss(), epochs = 10)"
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "KVgwGtDHgDlT",
|
||||
"outputId": "2447833f-f0e3-4566-c33d-addbfe2f451d"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Epoch: 1: loss=5.664632366860172\n",
|
||||
"Epoch: 2: loss=5.632101973960962\n",
|
||||
"Epoch: 3: loss=5.610399051405015\n",
|
||||
"Epoch: 4: loss=5.594621561080262\n",
|
||||
"Epoch: 5: loss=5.582538017415446\n",
|
||||
"Epoch: 6: loss=5.572900234519603\n",
|
||||
"Epoch: 7: loss=5.564951676341915\n",
|
||||
"Epoch: 8: loss=5.558288112064614\n",
|
||||
"Epoch: 9: loss=5.552576955031129\n",
|
||||
"Epoch: 10: loss=5.547634165194347\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"5.547634165194347"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 16
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "W8u2qXZmaHVd"
|
||||
},
|
||||
"source": [
|
||||
"## 尝试使用 Word2Vec\n",
|
||||
"\n",
|
||||
"为了使用 Word2Vec，让我们提取与词汇表中所有单词对应的向量：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "r8TatcXjkU_t"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"vectors = torch.stack([embedder(torch.tensor(vocab[s])) for s in vocab.itos], 0)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "3OcX21UOaHVd"
|
||||
},
|
||||
"source": [
|
||||
"让我们看看，例如，单词**Paris**是如何编码成一个向量的：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "bz6tAeLzieXp",
|
||||
"outputId": "5b20850e-4342-45e9-f840-cfac2b4d61d8"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"tensor([-0.0915, 2.1224, -0.0281, -0.6819, 1.1219, 0.6458, -1.3704, -1.3314,\n",
|
||||
" -1.1437, 0.4496, 0.2301, -0.3515, -0.8485, 1.0481, 0.4386, -0.8949,\n",
|
||||
" 0.5644, 1.0939, -2.5096, 3.2949, -0.2601, -0.8640, 0.1421, -0.0804,\n",
|
||||
" -0.5083, -1.0560, 0.9753, -0.5949, -1.6046, 0.5774],\n",
|
||||
" grad_fn=<EmbeddingBackward>)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"paris_vec = embedder(torch.tensor(vocab['paris']))\n",
|
||||
"print(paris_vec)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "pHTJlaeYaHVd"
|
||||
},
|
||||
"source": [
|
||||
"使用Word2Vec寻找同义词非常有趣。以下函数将返回与给定输入最接近的`n`个单词。为了找到它们，我们计算$|w_i - v|$的范数，其中$v$是与输入单词对应的向量，$w_i$是词汇表中第$i$个单词的编码。然后我们对数组进行排序，并使用`argsort`返回对应的索引，取列表的前`n`个元素，这些元素编码了词汇表中最接近单词的位置。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "NlZyi-_olFar",
|
||||
"outputId": "b5dbb163-88c4-4d5a-eaf2-6751f700e98c"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['microsoft', 'quoted', 'lp', 'rate', 'top']"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 56
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def close_words(x, n = 5):\n",
|
||||
" vec = embedder(torch.tensor(vocab[x]))\n",
|
||||
" top5 = np.linalg.norm(vectors.detach().numpy() - vec.detach().numpy(), axis = 1).argsort()[:n]\n",
|
||||
" return [ vocab.itos[x] for x in top5 ]\n",
|
||||
"\n",
|
||||
"close_words('microsoft')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "-dQq7xeAln0U",
|
||||
"outputId": "66f768c3-c248-4bfd-ce4f-c8ffc6d0dd0d"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['basketball', 'lot', 'sinai', 'states', 'healthdaynews']"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 51
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"close_words('basketball')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "fJXqK26b29sa",
|
||||
"outputId": "78f0baba-ffd0-485a-dd87-0a12bedfd7fa"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['funds', 'travel', 'sydney', 'japan', 'business']"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 77
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"close_words('funds')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "My0VeTDd3Ji8"
|
||||
},
|
||||
"source": [
|
||||
"## 要点\n",
|
||||
"\n",
|
||||
"通过使用诸如CBoW这样的巧妙技术，我们可以训练Word2Vec模型。你也可以尝试训练skip-gram模型，该模型通过给定中心词来预测邻近词，看看它的表现如何。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"collapsed_sections": [],
|
||||
"name": "CBoW-PyTorch.ipynb",
|
||||
"provenance": []
|
||||
},
|
||||
"interpreter": {
|
||||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.12 ('py38')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"gpuClass": "standard",
|
||||
"coopTranslator": {
|
||||
"original_hash": "36df28efe3fe40b6fb0a7fa48fe3ea82",
|
||||
"translation_date": "2025-08-31T10:36:21+00:00",
|
||||
"source_file": "lessons/5-NLP/15-LanguageModeling/CBoW-PyTorch.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,45 @@
|
|||
# 语言建模
|
||||
|
||||
语义嵌入（如 Word2Vec 和 GloVe）实际上是迈向**语言建模**的第一步——创建某种能够*理解*（或*表示*）语言本质的模型。
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/29)
|
||||
|
||||
语言建模的核心思想是通过无监督方式在未标注的数据集上进行训练。这非常重要，因为我们拥有大量未标注的文本，而标注文本的数量总是受到标注工作量的限制。通常，我们可以构建能够**预测文本中缺失单词**的语言模型，因为在文本中随机遮盖一个单词并将其作为训练样本是非常容易的。
|
||||
|
||||
## 嵌入训练
|
||||
|
||||
在之前的例子中，我们使用了预训练的语义嵌入，但了解这些嵌入是如何训练的也很有趣。有几种可能的思路可以用来训练嵌入：
|
||||
|
||||
* **N-Gram**语言建模，通过查看前 N 个标记（N-gram）来预测一个标记。
|
||||
* **连续词袋模型** (CBoW)，通过预测标记序列 $W_{-N}$, ..., $W_N$ 中的中间标记 $W_0$。
|
||||
* **Skip-gram**，通过中间标记 $W_0$ 来预测一组邻近标记 {$W_{-N},\dots, W_{-1}, W_1,\dots, W_N$}。
|
||||
|
||||
|
||||
|
||||
> 图片来源于[这篇论文](https://arxiv.org/pdf/1301.3781.pdf)
|
||||
|
||||
## ✍️ 示例笔记本：训练 CBoW 模型
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
* [使用 TensorFlow 训练 CBoW Word2Vec](CBoW-TF.ipynb)
|
||||
* [使用 PyTorch 训练 CBoW Word2Vec](CBoW-PyTorch.ipynb)
|
||||
|
||||
## 总结
|
||||
|
||||
在上一节课中，我们看到单词嵌入的效果非常神奇！现在我们知道训练单词嵌入并不是一项非常复杂的任务，如果需要，我们应该能够为特定领域的文本训练自己的单词嵌入。
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/30)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
* [PyTorch 官方语言建模教程](https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html)。
|
||||
* [TensorFlow 官方 Word2Vec 模型训练教程](https://www.TensorFlow.org/tutorials/text/word2vec)。
|
||||
* 使用 **gensim** 框架训练最常用嵌入的简便方法详见[此文档](https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html)。
|
||||
|
||||
## 🚀 [作业：训练 Skip-Gram 模型](lab/README.md)
|
||||
|
||||
在实验中，我们挑战你修改本课的代码以训练 Skip-Gram 模型而不是 CBoW。[阅读详情](lab/README.md)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# 训练 Skip-Gram 模型
|
||||
|
||||
来自 [AI for Beginners Curriculum](https://github.com/microsoft/ai-for-beginners) 的实验任务。
|
||||
|
||||
## 任务
|
||||
|
||||
在本实验中，我们挑战你使用 Skip-Gram 技术训练 Word2Vec 模型。训练一个嵌入网络来预测 $N$ 个词宽的 Skip-Gram 窗口中的邻近词。你可以使用[本课中的代码](../../../../../../lessons/5-NLP/15-LanguageModeling/CBoW-TF.ipynb)，并稍作修改。
|
||||
|
||||
## 数据集
|
||||
|
||||
你可以使用任何书籍。你可以在 [Project Gutenberg](https://www.gutenberg.org/) 找到许多免费文本，例如，这里有一个直接链接到刘易斯·卡罗尔的《爱丽丝梦游仙境》：[Alice's Adventures in Wonderland](https://www.gutenberg.org/files/11/11-0.txt)。或者，你可以使用莎士比亚的戏剧，以下代码可以获取：
|
||||
|
||||
```python
|
||||
path_to_file = tf.keras.utils.get_file(
|
||||
'shakespeare.txt',
|
||||
'https://storage.googleapis.com/download.tensorflow.org/data/shakespeare.txt')
|
||||
text = open(path_to_file, 'rb').read().decode(encoding='utf-8')
|
||||
```
|
||||
|
||||
## 探索！
|
||||
|
||||
如果你有时间并希望深入研究这个主题，可以尝试探索以下内容：
|
||||
|
||||
* 嵌入大小如何影响结果？
|
||||
* 不同的文本风格如何影响结果？
|
||||
* 选择几组非常不同类型的单词及其同义词，获取它们的向量表示，应用 PCA 将维度降到 2，并在二维空间中绘制它们。你能发现任何模式吗？
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,86 @@
|
|||
# 循环神经网络
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/31)
|
||||
|
||||
在之前的章节中，我们使用了丰富的文本语义表示，并在嵌入层之上使用了一个简单的线性分类器。这种架构能够捕捉句子中词语的整体意义，但它没有考虑到词语的**顺序**，因为嵌入层上的聚合操作会丢失原始文本中的顺序信息。由于这些模型无法建模词语的顺序，它们无法解决更复杂或更模糊的任务，例如文本生成或问答。
|
||||
|
||||
为了捕捉文本序列的意义，我们需要使用另一种神经网络架构，称为**循环神经网络**（Recurrent Neural Network，RNN）。在 RNN 中，我们将句子逐个符号输入网络，网络会生成某种**状态**，然后将该状态与下一个符号一起再次输入网络。
|
||||
|
||||
|
||||
|
||||
> 图片由作者提供
|
||||
|
||||
给定输入序列的标记 X<sub>0</sub>,...,X<sub>n</sub>，RNN 会创建一个神经网络块的序列，并通过反向传播对该序列进行端到端训练。每个网络块以 (X<sub>i</sub>,S<sub>i</sub>) 为输入，并生成 S<sub>i+1</sub> 作为结果。最终状态 S<sub>n</sub> 或 (输出 Y<sub>n</sub>) 会进入线性分类器以生成结果。所有网络块共享相同的权重，并通过一次反向传播训练端到端。
|
||||
|
||||
由于状态向量 S<sub>0</sub>,...,S<sub>n</sub> 会通过网络传递，RNN 能够学习词语之间的顺序依赖关系。例如，当单词 *not* 出现在序列中的某处时，它可以学习在状态向量中否定某些元素，从而实现否定。
|
||||
|
||||
> ✅ 由于上图中所有 RNN 块的权重是共享的，因此可以将整个图表示为一个带有循环反馈的单块（右侧），将网络的输出状态反馈到输入。
|
||||
|
||||
## RNN 单元的结构
|
||||
|
||||
让我们看看一个简单的 RNN 单元是如何组织的。它接受前一个状态 S<sub>i-1</sub> 和当前符号 X<sub>i</sub> 作为输入，并生成输出状态 S<sub>i</sub>（有时我们也对其他输出 Y<sub>i</sub> 感兴趣，例如在生成网络中）。
|
||||
|
||||
一个简单的 RNN 单元内部有两个权重矩阵：一个用于转换输入符号（我们称之为 W），另一个用于转换输入状态（H）。在这种情况下，网络的输出计算公式为 σ(W×X<sub>i</sub>+H×S<sub>i-1</sub>+b)，其中 σ 是激活函数，b 是额外的偏置。
|
||||
|
||||
<img alt="RNN 单元结构" src="../../../../../translated_images/zh-CN/rnn-anatomy.79ee3f3920b3294b.webp" width="50%"/>
|
||||
|
||||
> 图片由作者提供
|
||||
|
||||
在许多情况下，输入标记会在进入 RNN 之前通过嵌入层以降低维度。在这种情况下，如果输入向量的维度为 *emb_size*，状态向量的维度为 *hid_size*，那么 W 的大小为 *emb_size*×*hid_size*，H 的大小为 *hid_size*×*hid_size*。
|
||||
|
||||
## 长短时记忆网络（LSTM）
|
||||
|
||||
经典 RNN 的主要问题之一是所谓的**梯度消失**问题。由于 RNN 是通过一次反向传播端到端训练的，它很难将误差传播到网络的第一层，因此无法学习远距离标记之间的关系。解决这一问题的一种方法是通过使用所谓的**门控机制**引入**显式状态管理**。有两种著名的架构：**长短时记忆网络**（Long Short Term Memory，LSTM）和**门控循环单元**（Gated Relay Unit，GRU）。
|
||||
|
||||
|
||||
|
||||
> 图片来源待定
|
||||
|
||||
LSTM 网络的组织方式与 RNN 类似，但有两个状态会从层到层传递：实际状态 C 和隐藏向量 H。在每个单元中，隐藏向量 H<sub>i</sub> 与输入 X<sub>i</sub> 连接在一起，它们通过**门控机制**控制状态 C 的变化。每个门控机制都是一个带有 sigmoid 激活（输出范围为 [0,1]）的神经网络，可以通过与状态向量相乘来视为逐位掩码。以下是这些门控机制（从左到右）：
|
||||
|
||||
* **遗忘门**：接收隐藏向量并决定状态向量 C 的哪些部分需要遗忘，哪些需要保留。
|
||||
* **输入门**：从输入和隐藏向量中提取信息并插入状态。
|
||||
* **输出门**：通过带有 *tanh* 激活的线性层转换状态，然后使用隐藏向量 H<sub>i</sub> 选择状态的某些部分以生成新状态 C<sub>i+1</sub>。
|
||||
|
||||
状态 C 的各个组成部分可以被视为一些可以打开或关闭的标志。例如，当我们在序列中遇到名字 *Alice* 时，我们可能会假设它指代一个女性角色，并在状态中设置一个标志，表示句子中有一个女性名词。当我们进一步遇到短语 *and Tom* 时，我们会设置一个标志，表示句子中有复数名词。因此，通过操控状态，我们可以跟踪句子部分的语法属性。
|
||||
|
||||
> ✅ 理解 LSTM 内部机制的一个优秀资源是 Christopher Olah 的这篇文章 [Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)。
|
||||
|
||||
## 双向和多层 RNN
|
||||
|
||||
我们已经讨论了单向运行的循环网络，从序列的开头到结尾。这看起来很自然，因为它类似于我们阅读和听语音的方式。然而，由于在许多实际情况下我们可以随机访问输入序列，因此同时进行双向循环计算可能更有意义。这种网络称为**双向 RNN**。在处理双向网络时，我们需要两个隐藏状态向量，每个方向一个。
|
||||
|
||||
循环网络，无论是单向还是双向，都能捕捉序列中的某些模式，并将其存储到状态向量中或传递到输出中。与卷积网络类似，我们可以在第一层之上构建另一层循环网络，以捕捉更高级的模式，并从第一层提取的低级模式中构建。这引出了**多层 RNN** 的概念，它由两个或更多循环网络组成，其中前一层的输出作为输入传递到下一层。
|
||||
|
||||
|
||||
|
||||
*图片来自 Fernando López 的[这篇精彩文章](https://towardsdatascience.com/from-a-lstm-cell-to-a-multilayer-lstm-network-with-pytorch-2899eb5696f3)*
|
||||
|
||||
## ✍️ 练习：嵌入
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
* [使用 PyTorch 的 RNN](RNNPyTorch.ipynb)
|
||||
* [使用 TensorFlow 的 RNN](RNNTF.ipynb)
|
||||
|
||||
## 总结
|
||||
|
||||
在本单元中，我们了解到 RNN 可用于序列分类，但实际上它们可以处理更多任务，例如文本生成、机器翻译等。我们将在下一单元中讨论这些任务。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
阅读一些关于 LSTM 的文献并思考它们的应用：
|
||||
|
||||
- [网格长短时记忆](https://arxiv.org/pdf/1507.01526v1.pdf)
|
||||
- [展示、关注和讲述：带视觉注意力的神经图像字幕生成](https://arxiv.org/pdf/1502.03044v2.pdf)
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/32)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
- Christopher Olah 的 [Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)。
|
||||
|
||||
## [作业：笔记本](assignment.md)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,479 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 循环神经网络\n",
|
||||
"\n",
|
||||
"在之前的模块中，我们使用了丰富的文本语义表示，并在嵌入的基础上使用了一个简单的线性分类器。这种架构能够捕捉句子中词语的整体意义，但它没有考虑词语的**顺序**，因为嵌入上的聚合操作会将原始文本中的顺序信息移除。由于这些模型无法建模词语的顺序，它们无法解决更复杂或更模糊的任务，例如文本生成或问答。\n",
|
||||
"\n",
|
||||
"为了捕捉文本序列的意义，我们需要使用另一种神经网络架构，称为**循环神经网络**（Recurrent Neural Network，RNN）。在 RNN 中，我们将句子逐个符号传递给网络，网络会生成某种**状态**，然后我们将该状态与下一个符号一起再次传递给网络。\n",
|
||||
"\n",
|
||||
"给定输入序列的标记 $X_0,\\dots,X_n$，RNN 会创建一个神经网络块的序列，并通过反向传播对该序列进行端到端训练。每个网络块以 $(X_i,S_i)$ 作为输入，并生成 $S_{i+1}$ 作为结果。最终状态 $S_n$ 或输出 $X_n$ 会传递给线性分类器以生成结果。所有网络块共享相同的权重，并通过一次反向传播训练端到端。\n",
|
||||
"\n",
|
||||
"由于状态向量 $S_0,\\dots,S_n$ 会通过网络传递，RNN 能够学习词语之间的顺序依赖关系。例如，当单词 *not* 出现在序列中的某处时，它可以学习在状态向量中否定某些元素，从而实现否定。\n",
|
||||
"\n",
|
||||
"> 由于图片中所有 RNN 块的权重是共享的，因此同一张图片可以表示为一个带有循环反馈回路的单块（右侧），该回路将网络的输出状态传回输入。\n",
|
||||
"\n",
|
||||
"接下来我们看看循环神经网络如何帮助我们对新闻数据集进行分类。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Loading dataset...\n",
|
||||
"Building vocab...\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"import torchtext\n",
|
||||
"from torchnlp import *\n",
|
||||
"train_dataset, test_dataset, classes, vocab = load_dataset()\n",
|
||||
"vocab_size = len(vocab)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 简单 RNN 分类器\n",
|
||||
"\n",
|
||||
"对于简单的 RNN，每个循环单元都是一个简单的线性网络，它接收拼接后的输入向量和状态向量，并生成一个新的状态向量。PyTorch 使用 `RNNCell` 类表示这个单元，并将由这些单元组成的网络表示为 `RNN` 层。\n",
|
||||
"\n",
|
||||
"为了定义一个 RNN 分类器，我们首先会应用一个嵌入层来降低输入词汇的维度，然后在其上添加一个 RNN 层：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class RNNClassifier(torch.nn.Module):\n",
|
||||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||||
" super().__init__()\n",
|
||||
" self.hidden_dim = hidden_dim\n",
|
||||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||||
" self.rnn = torch.nn.RNN(embed_dim,hidden_dim,batch_first=True)\n",
|
||||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||||
"\n",
|
||||
" def forward(self, x):\n",
|
||||
" batch_size = x.size(0)\n",
|
||||
" x = self.embedding(x)\n",
|
||||
" x,h = self.rnn(x)\n",
|
||||
" return self.fc(x.mean(dim=1))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意：** 为了简化操作，我们这里使用了未训练的嵌入层，但如果想要更好的效果，可以使用预训练的嵌入层，例如 Word2Vec 或 GloVe 嵌入，这在前一单元中已有描述。为了更好地理解，您可能需要将此代码调整为使用预训练的嵌入。\n",
|
||||
"\n",
|
||||
"在我们的例子中，我们将使用填充数据加载器，因此每个批次将包含若干长度相同的填充序列。RNN 层将接收嵌入张量的序列，并生成两个输出：\n",
|
||||
"* $x$ 是每一步 RNN 单元输出的序列\n",
|
||||
"* $h$ 是序列最后一个元素的最终隐藏状态\n",
|
||||
"\n",
|
||||
"然后我们应用一个全连接的线性分类器来获得类别数量。\n",
|
||||
"\n",
|
||||
"> **注意：** RNN 的训练相当困难，因为一旦 RNN 单元沿着序列长度展开，反向传播中涉及的层数会非常多。因此，我们需要选择较小的学习率，并在更大的数据集上训练网络以获得良好的结果。这可能需要相当长的时间，因此建议使用 GPU。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3200: acc=0.3090625\n",
|
||||
"6400: acc=0.38921875\n",
|
||||
"9600: acc=0.4590625\n",
|
||||
"12800: acc=0.511953125\n",
|
||||
"16000: acc=0.5506875\n",
|
||||
"19200: acc=0.57921875\n",
|
||||
"22400: acc=0.6070089285714285\n",
|
||||
"25600: acc=0.6304296875\n",
|
||||
"28800: acc=0.6484027777777778\n",
|
||||
"32000: acc=0.66509375\n",
|
||||
"35200: acc=0.6790056818181818\n",
|
||||
"38400: acc=0.6929166666666666\n",
|
||||
"41600: acc=0.7035817307692308\n",
|
||||
"44800: acc=0.7137276785714286\n",
|
||||
"48000: acc=0.72225\n",
|
||||
"51200: acc=0.73001953125\n",
|
||||
"54400: acc=0.7372794117647059\n",
|
||||
"57600: acc=0.7436631944444444\n",
|
||||
"60800: acc=0.7503947368421052\n",
|
||||
"64000: acc=0.75634375\n",
|
||||
"67200: acc=0.7615773809523809\n",
|
||||
"70400: acc=0.7662642045454545\n",
|
||||
"73600: acc=0.7708423913043478\n",
|
||||
"76800: acc=0.7751822916666666\n",
|
||||
"80000: acc=0.7790625\n",
|
||||
"83200: acc=0.7825\n",
|
||||
"86400: acc=0.7858564814814815\n",
|
||||
"89600: acc=0.7890513392857142\n",
|
||||
"92800: acc=0.7920474137931034\n",
|
||||
"96000: acc=0.7952708333333334\n",
|
||||
"99200: acc=0.7982258064516129\n",
|
||||
"102400: acc=0.80099609375\n",
|
||||
"105600: acc=0.8037594696969697\n",
|
||||
"108800: acc=0.8060569852941176\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=padify, shuffle=True)\n",
|
||||
"net = RNNClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||||
"train_epoch(net,train_loader, lr=0.001)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 长短期记忆网络 (LSTM)\n",
|
||||
"\n",
|
||||
"经典 RNN 的主要问题之一是所谓的 **梯度消失** 问题。由于 RNN 是通过一次反向传播端到端训练的，因此在将误差传播到网络的第一层时会遇到困难，从而无法学习远距离的词元之间的关系。解决这一问题的一种方法是通过使用所谓的 **门控机制** 引入 **显式状态管理**。最著名的两种架构是：**长短期记忆网络** (LSTM) 和 **门控循环单元** (GRU)。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"LSTM 网络的组织方式与 RNN 类似，但有两个状态会从层传递到层：实际状态 $c$ 和隐藏向量 $h$。在每个单元中，隐藏向量 $h_i$ 与输入 $x_i$ 连接在一起，并通过 **门控机制** 控制状态 $c$ 的变化。每个门都是一个带有 sigmoid 激活函数的神经网络（输出范围为 $[0,1]$），可以通过与状态向量相乘来理解为逐位掩码。以下是这些门（如上图从左到右）：\n",
|
||||
"* **遗忘门** 接收隐藏向量并决定状态向量 $c$ 的哪些部分需要遗忘，哪些需要保留。\n",
|
||||
"* **输入门** 从输入和隐藏向量中提取一些信息，并将其插入状态中。\n",
|
||||
"* **输出门** 通过某种带有 $\\tanh$ 激活的线性层转换状态，然后使用隐藏向量 $h_i$ 选择状态的某些部分以生成新的状态 $c_{i+1}$。\n",
|
||||
"\n",
|
||||
"状态 $c$ 的各个组成部分可以被视为一些可以打开或关闭的标志。例如，当我们在序列中遇到名字 *Alice* 时，我们可能会假设它指代女性角色，并在状态中设置一个标志，表示句子中有女性名词。当我们进一步遇到短语 *and Tom* 时，我们会设置一个标志，表示句子中有复数名词。因此，通过操控状态，我们可以追踪句子部分的语法属性。\n",
|
||||
"\n",
|
||||
"> **Note**: 理解 LSTM 内部结构的一个优秀资源是 Christopher Olah 的这篇文章 [Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)。\n",
|
||||
"\n",
|
||||
"虽然 LSTM 单元的内部结构看起来很复杂，但 PyTorch 将其实现隐藏在 `LSTMCell` 类中，并提供了 `LSTM` 对象来表示整个 LSTM 层。因此，LSTM 分类器的实现与我们之前看到的简单 RNN 非常相似：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class LSTMClassifier(torch.nn.Module):\n",
|
||||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||||
" super().__init__()\n",
|
||||
" self.hidden_dim = hidden_dim\n",
|
||||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||||
" self.embedding.weight.data = torch.randn_like(self.embedding.weight.data)-0.5\n",
|
||||
" self.rnn = torch.nn.LSTM(embed_dim,hidden_dim,batch_first=True)\n",
|
||||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||||
"\n",
|
||||
" def forward(self, x):\n",
|
||||
" batch_size = x.size(0)\n",
|
||||
" x = self.embedding(x)\n",
|
||||
" x,(h,c) = self.rnn(x)\n",
|
||||
" return self.fc(h[-1])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3200: acc=0.259375\n",
|
||||
"6400: acc=0.25859375\n",
|
||||
"9600: acc=0.26177083333333334\n",
|
||||
"12800: acc=0.2784375\n",
|
||||
"16000: acc=0.313\n",
|
||||
"19200: acc=0.3528645833333333\n",
|
||||
"22400: acc=0.3965625\n",
|
||||
"25600: acc=0.4385546875\n",
|
||||
"28800: acc=0.4752777777777778\n",
|
||||
"32000: acc=0.505375\n",
|
||||
"35200: acc=0.5326704545454546\n",
|
||||
"38400: acc=0.5557552083333334\n",
|
||||
"41600: acc=0.5760817307692307\n",
|
||||
"44800: acc=0.5954910714285714\n",
|
||||
"48000: acc=0.6118333333333333\n",
|
||||
"51200: acc=0.62681640625\n",
|
||||
"54400: acc=0.6404779411764706\n",
|
||||
"57600: acc=0.6520138888888889\n",
|
||||
"60800: acc=0.662828947368421\n",
|
||||
"64000: acc=0.673546875\n",
|
||||
"67200: acc=0.6831547619047619\n",
|
||||
"70400: acc=0.6917897727272727\n",
|
||||
"73600: acc=0.6997146739130434\n",
|
||||
"76800: acc=0.707109375\n",
|
||||
"80000: acc=0.714075\n",
|
||||
"83200: acc=0.7209134615384616\n",
|
||||
"86400: acc=0.727037037037037\n",
|
||||
"89600: acc=0.7326674107142858\n",
|
||||
"92800: acc=0.7379633620689655\n",
|
||||
"96000: acc=0.7433645833333333\n",
|
||||
"99200: acc=0.7479032258064516\n",
|
||||
"102400: acc=0.752119140625\n",
|
||||
"105600: acc=0.7562405303030303\n",
|
||||
"108800: acc=0.76015625\n",
|
||||
"112000: acc=0.7641339285714286\n",
|
||||
"115200: acc=0.7677777777777778\n",
|
||||
"118400: acc=0.7711233108108108\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(0.03487814127604167, 0.7728)"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"net = LSTMClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||||
"train_epoch(net,train_loader, lr=0.001)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 填充序列\n",
|
||||
"\n",
|
||||
"在我们的示例中，我们需要用零向量填充小批量中的所有序列。虽然这会导致一些内存浪费，但对于 RNN 来说，更重要的是为填充的输入项创建额外的 RNN 单元，这些单元参与训练，但并不包含任何重要的输入信息。更好的方法是仅根据实际序列长度来训练 RNN。\n",
|
||||
"\n",
|
||||
"为此，PyTorch 引入了一种特殊的填充序列存储格式。假设我们有一个填充的小批量输入，如下所示：\n",
|
||||
"```\n",
|
||||
"[[1,2,3,4,5],\n",
|
||||
" [6,7,8,0,0],\n",
|
||||
" [9,0,0,0,0]]\n",
|
||||
"```\n",
|
||||
"这里的 0 表示填充值，输入序列的实际长度向量为 `[5,3,1]`。\n",
|
||||
"\n",
|
||||
"为了有效地用填充序列训练 RNN，我们希望首先用较大的小批量（`[1,6,9]`）开始训练第一组 RNN 单元，然后结束第三个序列的处理，并继续用较小的小批量（`[2,7]`，`[3,8]`）进行训练，依此类推。因此，填充序列被表示为一个向量——在我们的例子中是 `[1,6,9,2,7,3,8,4,5]`，以及长度向量（`[5,3,1]`），通过这些信息我们可以轻松地重建原始填充的小批量。\n",
|
||||
"\n",
|
||||
"为了生成填充序列，我们可以使用 `torch.nn.utils.rnn.pack_padded_sequence` 函数。所有循环层，包括 RNN、LSTM 和 GRU，都支持将填充序列作为输入，并生成填充序列的输出，这些输出可以通过 `torch.nn.utils.rnn.pad_packed_sequence` 解码。\n",
|
||||
"\n",
|
||||
"为了能够生成填充序列，我们需要将长度向量传递给网络，因此我们需要一个不同的函数来准备小批量：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def pad_length(b):\n",
|
||||
" # build vectorized sequence\n",
|
||||
" v = [encode(x[1]) for x in b]\n",
|
||||
" # compute max length of a sequence in this minibatch and length sequence itself\n",
|
||||
" len_seq = list(map(len,v))\n",
|
||||
" l = max(len_seq)\n",
|
||||
" return ( # tuple of three tensors - labels, padded features, length sequence\n",
|
||||
" torch.LongTensor([t[0]-1 for t in b]),\n",
|
||||
" torch.stack([torch.nn.functional.pad(torch.tensor(t),(0,l-len(t)),mode='constant',value=0) for t in v]),\n",
|
||||
" torch.tensor(len_seq)\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"train_loader_len = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=pad_length, shuffle=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"实际的网络与上面的 `LSTMClassifier` 非常相似，但 `forward` 传递会接收填充后的小批量数据以及序列长度的向量。在计算嵌入后，我们计算打包序列，将其传递给 LSTM 层，然后将结果解包回来。\n",
|
||||
"\n",
|
||||
"> **注意**: 实际上我们并不使用解包后的结果 `x`，因为我们在后续计算中使用的是隐藏层的输出。因此，我们可以完全从代码中移除解包操作。之所以将其保留在这里，是为了方便您在需要使用网络输出进行进一步计算时能够轻松修改代码。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class LSTMPackClassifier(torch.nn.Module):\n",
|
||||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||||
" super().__init__()\n",
|
||||
" self.hidden_dim = hidden_dim\n",
|
||||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||||
" self.embedding.weight.data = torch.randn_like(self.embedding.weight.data)-0.5\n",
|
||||
" self.rnn = torch.nn.LSTM(embed_dim,hidden_dim,batch_first=True)\n",
|
||||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||||
"\n",
|
||||
" def forward(self, x, lengths):\n",
|
||||
" batch_size = x.size(0)\n",
|
||||
" x = self.embedding(x)\n",
|
||||
" pad_x = torch.nn.utils.rnn.pack_padded_sequence(x,lengths,batch_first=True,enforce_sorted=False)\n",
|
||||
" pad_x,(h,c) = self.rnn(pad_x)\n",
|
||||
" x, _ = torch.nn.utils.rnn.pad_packed_sequence(pad_x,batch_first=True)\n",
|
||||
" return self.fc(h[-1])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"3200: acc=0.285625\n",
|
||||
"6400: acc=0.33359375\n",
|
||||
"9600: acc=0.3876041666666667\n",
|
||||
"12800: acc=0.44078125\n",
|
||||
"16000: acc=0.4825\n",
|
||||
"19200: acc=0.5235416666666667\n",
|
||||
"22400: acc=0.5559821428571429\n",
|
||||
"25600: acc=0.58609375\n",
|
||||
"28800: acc=0.6116666666666667\n",
|
||||
"32000: acc=0.63340625\n",
|
||||
"35200: acc=0.6525284090909091\n",
|
||||
"38400: acc=0.668515625\n",
|
||||
"41600: acc=0.6822596153846154\n",
|
||||
"44800: acc=0.6948214285714286\n",
|
||||
"48000: acc=0.7052708333333333\n",
|
||||
"51200: acc=0.71521484375\n",
|
||||
"54400: acc=0.7239889705882353\n",
|
||||
"57600: acc=0.7315277777777778\n",
|
||||
"60800: acc=0.7388486842105263\n",
|
||||
"64000: acc=0.74571875\n",
|
||||
"67200: acc=0.7518303571428572\n",
|
||||
"70400: acc=0.7576988636363636\n",
|
||||
"73600: acc=0.7628940217391305\n",
|
||||
"76800: acc=0.7681510416666667\n",
|
||||
"80000: acc=0.7728125\n",
|
||||
"83200: acc=0.7772235576923077\n",
|
||||
"86400: acc=0.7815393518518519\n",
|
||||
"89600: acc=0.7857700892857142\n",
|
||||
"92800: acc=0.7895043103448276\n",
|
||||
"96000: acc=0.7930520833333333\n",
|
||||
"99200: acc=0.7959072580645161\n",
|
||||
"102400: acc=0.798994140625\n",
|
||||
"105600: acc=0.802064393939394\n",
|
||||
"108800: acc=0.8051378676470589\n",
|
||||
"112000: acc=0.8077857142857143\n",
|
||||
"115200: acc=0.8104600694444445\n",
|
||||
"118400: acc=0.8128293918918919\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(0.029785829671223958, 0.8138166666666666)"
|
||||
]
|
||||
},
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"net = LSTMPackClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||||
"train_epoch_emb(net,train_loader_len, lr=0.001,use_pack_sequence=True)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意：** 您可能已经注意到我们传递给训练函数的参数 `use_pack_sequence`。目前，`pack_padded_sequence` 函数要求长度序列张量位于 CPU 设备上，因此训练函数需要避免在训练时将长度序列数据移动到 GPU。您可以查看 [`torchnlp.py`](../../../../../lessons/5-NLP/16-RNN/torchnlp.py) 文件中 `train_emb` 函数的实现。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 双向和多层 RNN\n",
|
||||
"\n",
|
||||
"在我们的示例中，所有的循环网络都是单向运行的，从序列的开始到结束。这看起来很自然，因为它类似于我们阅读和听语音的方式。然而，在许多实际情况下，我们可以随机访问输入序列，因此在两个方向上运行循环计算可能更有意义。这类网络称为**双向** RNN，可以通过向 RNN/LSTM/GRU 构造函数传递 `bidirectional=True` 参数来创建。\n",
|
||||
"\n",
|
||||
"在处理双向网络时，我们需要两个隐藏状态向量，每个方向一个。PyTorch 将这些向量编码为一个大小加倍的向量，这非常方便，因为通常会将生成的隐藏状态传递给全连接线性层，只需在创建该层时考虑到大小的增加即可。\n",
|
||||
"\n",
|
||||
"无论是单向还是双向的循环网络，都能捕捉序列中的某些模式，并将其存储到状态向量中或传递到输出中。与卷积网络类似，我们可以在第一层循环网络之上构建另一层循环网络，以捕捉更高层次的模式，这些模式是由第一层提取的低层次模式构建而成的。这引出了**多层 RNN**的概念，它由两个或更多的循环网络组成，其中上一层的输出作为下一层的输入。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"*图片来源于 [Fernando López 的这篇精彩文章](https://towardsdatascience.com/from-a-lstm-cell-to-a-multilayer-lstm-network-with-pytorch-2899eb5696f3)*\n",
|
||||
"\n",
|
||||
"PyTorch 使构建此类网络变得非常简单，因为只需向 RNN/LSTM/GRU 构造函数传递 `num_layers` 参数，就可以自动构建多层循环网络。这也意味着隐藏状态向量的大小会按比例增加，在处理循环层的输出时需要考虑这一点。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## RNN用于其他任务\n",
|
||||
"\n",
|
||||
"在本单元中，我们已经了解到RNN可以用于序列分类，但实际上，它还可以处理更多任务，例如文本生成、机器翻译等。我们将在下一单元中讨论这些任务。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。虽然我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于重要信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.12 ('py38')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "522ee52ae3d5ae933e283286254e9a55",
|
||||
"translation_date": "2025-08-31T10:49:43+00:00",
|
||||
"source_file": "lessons/5-NLP/16-RNN/RNNPyTorch.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,460 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 循环神经网络\n",
|
||||
"\n",
|
||||
"在上一模块中，我们讨论了文本的丰富语义表示。我们使用的架构能够捕捉句子中词语的聚合意义，但它没有考虑词语的**顺序**，因为嵌入后的聚合操作会丢失原始文本中的这一信息。由于这些模型无法表示词语的顺序，它们无法解决更复杂或更模糊的任务，例如文本生成或问答。\n",
|
||||
"\n",
|
||||
"为了捕捉文本序列的意义，我们将使用一种称为**循环神经网络**（RNN）的神经网络架构。在使用 RNN 时，我们会将句子逐个标记传递给网络，网络会生成某种**状态**，然后我们将该状态与下一个标记一起再次传递给网络。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"给定输入标记序列 $X_0,\\dots,X_n$，RNN 会创建一系列神经网络块，并通过反向传播对这一序列进行端到端训练。每个网络块以 $(X_i,S_i)$ 为输入，并生成 $S_{i+1}$ 作为结果。最终状态 $S_n$ 或输出 $Y_n$ 会进入线性分类器以生成结果。所有网络块共享相同的权重，并通过一次反向传播进行端到端训练。\n",
|
||||
"\n",
|
||||
"> 上图展示了循环神经网络的展开形式（左侧）和更紧凑的循环表示形式（右侧）。需要注意的是，所有 RNN 单元都具有相同的**可共享权重**。\n",
|
||||
"\n",
|
||||
"由于状态向量 $S_0,\\dots,S_n$ 会通过网络传递，RNN 能够学习词语之间的顺序依赖关系。例如，当单词 *not* 出现在序列中的某处时，它可以学习在状态向量中否定某些元素。\n",
|
||||
"\n",
|
||||
"在内部，每个 RNN 单元包含两个权重矩阵：$W_H$ 和 $W_I$，以及偏置 $b$。在每个 RNN 步骤中，给定输入 $X_i$ 和输入状态 $S_i$，输出状态计算公式为 $S_{i+1} = f(W_H\\times S_i + W_I\\times X_i+b)$，其中 $f$ 是一个激活函数（通常为 $\\tanh$）。\n",
|
||||
"\n",
|
||||
"> 对于像文本生成（我们将在下一单元中讨论）或机器翻译这样的问题，我们还希望在每个 RNN 步骤中获得一些输出值。在这种情况下，还会有另一个矩阵 $W_O$，输出计算公式为 $Y_i=f(W_O\\times S_i+b_O)$。\n",
|
||||
"\n",
|
||||
"让我们看看循环神经网络如何帮助我们对新闻数据集进行分类。\n",
|
||||
"\n",
|
||||
"> 在沙盒环境中，我们需要运行以下单元以确保所需库已安装，并预取数据。如果您在本地运行，可以跳过以下单元。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"!{sys.executable} -m pip install --quiet tensorflow_datasets==4.4.0\n",
|
||||
"!cd ~ && wget -q -O - https://mslearntensorflowlp.blob.core.windows.net/data/tfds-ag-news.tgz | tar xz"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorflow import keras\n",
|
||||
"import tensorflow_datasets as tfds\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"# We are going to be training pretty large models. In order not to face errors, we need\n",
|
||||
"# to set tensorflow option to grow GPU memory allocation when required\n",
|
||||
"physical_devices = tf.config.list_physical_devices('GPU') \n",
|
||||
"if len(physical_devices)>0:\n",
|
||||
" tf.config.experimental.set_memory_growth(physical_devices[0], True)\n",
|
||||
"\n",
|
||||
"ds_train, ds_test = tfds.load('ag_news_subset').values()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"nteract": {
|
||||
"transient": {
|
||||
"deleting": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"在训练大型模型时，GPU 内存分配可能会成为一个问题。我们还可能需要尝试不同的 minibatch 大小，以便数据能够适应 GPU 内存，同时确保训练速度足够快。如果您在自己的 GPU 机器上运行此代码，可以尝试调整 minibatch 大小来加快训练速度。\n",
|
||||
"\n",
|
||||
"> **Note**: 某些版本的 NVidia 驱动程序已知在训练模型后不会释放内存。我们在这个 notebook 中运行了多个示例，这可能会在某些设置中导致内存耗尽，特别是如果您在同一个 notebook 中进行自己的实验时。如果在开始训练模型时遇到一些奇怪的错误，您可能需要重启 notebook 的内核。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"jupyter": {
|
||||
"outputs_hidden": false,
|
||||
"source_hidden": false
|
||||
},
|
||||
"nteract": {
|
||||
"transient": {
|
||||
"deleting": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"batch_size = 16\n",
|
||||
"embed_size = 64"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 简单RNN分类器\n",
|
||||
"\n",
|
||||
"对于简单RNN，每个循环单元都是一个简单的线性网络，它接收一个输入向量和状态向量，并生成一个新的状态向量。在Keras中，这可以通过`SimpleRNN`层来表示。\n",
|
||||
"\n",
|
||||
"虽然我们可以直接将独热编码的标记传递给RNN层，但这并不是一个好主意，因为它们的维度过高。因此，我们将使用嵌入层来降低词向量的维度，接着是一个RNN层，最后是一个`Dense`分类器。\n",
|
||||
"\n",
|
||||
"> **注意**: 在维度不是特别高的情况下，例如使用字符级标记化时，直接将独热编码的标记传递给RNN单元可能是合理的。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"sequential\"\n",
|
||||
"_________________________________________________________________\n",
|
||||
"Layer (type) Output Shape Param # \n",
|
||||
"=================================================================\n",
|
||||
"text_vectorization (TextVect (None, None) 0 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"embedding (Embedding) (None, None, 64) 1280000 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"simple_rnn (SimpleRNN) (None, 16) 1296 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"dense (Dense) (None, 4) 68 \n",
|
||||
"=================================================================\n",
|
||||
"Total params: 1,281,364\n",
|
||||
"Trainable params: 1,281,364\n",
|
||||
"Non-trainable params: 0\n",
|
||||
"_________________________________________________________________\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vocab_size = 20000\n",
|
||||
"\n",
|
||||
"vectorizer = keras.layers.experimental.preprocessing.TextVectorization(\n",
|
||||
" max_tokens=vocab_size,\n",
|
||||
" input_shape=(1,))\n",
|
||||
"\n",
|
||||
"model = keras.models.Sequential([\n",
|
||||
" vectorizer,\n",
|
||||
" keras.layers.Embedding(vocab_size, embed_size),\n",
|
||||
" keras.layers.SimpleRNN(16),\n",
|
||||
" keras.layers.Dense(4,activation='softmax')\n",
|
||||
"])\n",
|
||||
"\n",
|
||||
"model.summary()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意：** 这里我们为了简化使用了未训练的嵌入层，但为了获得更好的效果，可以使用上一单元中提到的 Word2Vec 预训练嵌入层。这是一个很好的练习，你可以尝试将代码改为使用预训练的嵌入层。\n",
|
||||
"\n",
|
||||
"现在让我们开始训练 RNN。一般来说，RNN 的训练难度较大，因为当 RNN 单元沿着序列长度展开时，参与反向传播的层数会非常多。因此，我们需要选择较小的学习率，并在更大的数据集上训练网络以获得良好的结果。这可能需要较长的时间，因此建议使用 GPU。\n",
|
||||
"\n",
|
||||
"为了加快速度，我们将仅使用新闻标题来训练 RNN 模型，省略描述部分。你可以尝试加入描述进行训练，看看是否能让模型成功训练。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training vectorizer\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def extract_title(x):\n",
|
||||
" return x['title']\n",
|
||||
"\n",
|
||||
"def tupelize_title(x):\n",
|
||||
" return (extract_title(x),x['label'])\n",
|
||||
"\n",
|
||||
"print('Training vectorizer')\n",
|
||||
"vectorizer.adapt(ds_train.take(2000).map(extract_title))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"7500/7500 [==============================] - 82s 11ms/step - loss: 0.6629 - acc: 0.7623 - val_loss: 0.5559 - val_acc: 0.7995\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7f3e0030d350>"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
||||
"model.fit(ds_train.map(tupelize_title).batch(batch_size),validation_data=ds_test.map(tupelize_title).batch(batch_size))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"nteract": {
|
||||
"transient": {
|
||||
"deleting": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"> **注意**，由于我们仅在新闻标题上进行训练，这里的准确性可能会较低。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 重新审视变量序列\n",
|
||||
"\n",
|
||||
"请记住，`TextVectorization` 层会自动使用填充标记对小批量中长度不一的序列进行填充。然而，这些填充标记也会参与训练，并可能使模型的收敛变得复杂。\n",
|
||||
"\n",
|
||||
"我们可以采取几种方法来减少填充的数量。其中一种方法是根据序列长度重新排序数据集，并按大小对所有序列进行分组。这可以通过使用 `tf.data.experimental.bucket_by_sequence_length` 函数来实现（参见[文档](https://www.tensorflow.org/api_docs/python/tf/data/experimental/bucket_by_sequence_length))。\n",
|
||||
"\n",
|
||||
"另一种方法是使用**掩码**。在 Keras 中，某些层支持额外的输入，用于指示在训练时应该考虑哪些标记。为了在模型中加入掩码，我们可以选择添加一个单独的 `Masking` 层（[文档](https://keras.io/api/layers/core_layers/masking/)），或者在 `Embedding` 层中指定 `mask_zero=True` 参数。\n",
|
||||
"\n",
|
||||
"> **Note**: 完成整个数据集的一次训练周期大约需要 5 分钟。如果你没有耐心，可以随时中断训练。你还可以通过在 `ds_train` 和 `ds_test` 数据集后添加 `.take(...)` 子句来限制用于训练的数据量。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"7500/7500 [==============================] - 371s 49ms/step - loss: 0.5401 - acc: 0.8079 - val_loss: 0.3780 - val_acc: 0.8822\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7f3dec118850>"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def extract_text(x):\n",
|
||||
" return x['title']+' '+x['description']\n",
|
||||
"\n",
|
||||
"def tupelize(x):\n",
|
||||
" return (extract_text(x),x['label'])\n",
|
||||
"\n",
|
||||
"model = keras.models.Sequential([\n",
|
||||
" vectorizer,\n",
|
||||
" keras.layers.Embedding(vocab_size,embed_size,mask_zero=True),\n",
|
||||
" keras.layers.SimpleRNN(16),\n",
|
||||
" keras.layers.Dense(4,activation='softmax')\n",
|
||||
"])\n",
|
||||
"\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(batch_size),validation_data=ds_test.map(tupelize).batch(batch_size))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在我们使用掩码技术，可以在整个标题和描述的数据集上训练模型。\n",
|
||||
"\n",
|
||||
"> **注意**: 你是否注意到我们一直在使用基于新闻标题训练的向量化器，而不是整篇文章的内容？这可能会导致某些词元被忽略，因此重新训练向量化器会更好。不过，这可能只会带来很小的影响，所以为了简化流程，我们将继续使用之前预训练的向量化器。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## LSTM: 长短期记忆\n",
|
||||
"\n",
|
||||
"RNN 的主要问题之一是**梯度消失**。RNN 可能会非常长，在反向传播过程中，梯度可能很难一直传播回网络的第一层。当这种情况发生时，网络无法学习远距离的标记之间的关系。为了解决这个问题，可以通过使用**门控机制**引入**显式状态管理**。最常见的两种引入门控机制的架构是**长短期记忆**（LSTM）和**门控循环单元**（GRU）。这里我们将讨论 LSTM。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"LSTM 网络的组织方式与 RNN 类似，但有两个状态会从一层传递到下一层：实际状态 $c$ 和隐藏向量 $h$。在每个单元中，隐藏向量 $h_{t-1}$ 与输入 $x_t$ 结合在一起，共同控制状态 $c_t$ 和输出 $h_t$ 的变化，这一过程通过**门控机制**完成。每个门都有 sigmoid 激活函数（输出范围为 $[0,1]$），可以将其视为与状态向量相乘的位掩码。LSTM 包含以下几种门（从上图从左到右）：\n",
|
||||
"* **遗忘门**：决定向量 $c_{t-1}$ 的哪些部分需要遗忘，哪些部分需要保留。\n",
|
||||
"* **输入门**：决定输入向量和前一个隐藏向量中的信息有多少需要被合并到状态向量中。\n",
|
||||
"* **输出门**：接收新的状态向量，并决定其哪些部分将用于生成新的隐藏向量 $h_t$。\n",
|
||||
"\n",
|
||||
"状态 $c$ 的各个组成部分可以被视为可以开关的标志。例如，当我们在序列中遇到名字 *Alice* 时，我们猜测它指的是一个女性，并在状态中设置一个标志，表示句子中有一个女性名词。当我们进一步遇到单词 *and Tom* 时，我们会设置另一个标志，表示句子中有复数名词。因此，通过操控状态，我们可以跟踪句子的语法属性。\n",
|
||||
"\n",
|
||||
"> **Note**: 这里有一个很棒的资源可以帮助理解 LSTM 的内部机制：[Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)，作者是 Christopher Olah。\n",
|
||||
"\n",
|
||||
"虽然 LSTM 单元的内部结构看起来很复杂，但 Keras 将这些实现隐藏在 `LSTM` 层中，因此在上面的示例中，我们只需要替换循环层即可：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"15000/15000 [==============================] - 188s 13ms/step - loss: 0.5692 - acc: 0.7916 - val_loss: 0.3441 - val_acc: 0.8870\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7f3d6af5c350>"
|
||||
]
|
||||
},
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = keras.models.Sequential([\n",
|
||||
" vectorizer,\n",
|
||||
" keras.layers.Embedding(vocab_size, embed_size),\n",
|
||||
" keras.layers.LSTM(8),\n",
|
||||
" keras.layers.Dense(4,activation='softmax')\n",
|
||||
"])\n",
|
||||
"\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(8),validation_data=ds_test.map(tupelize).batch(8))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 双向和多层 RNN\n",
|
||||
"\n",
|
||||
"在我们之前的示例中，循环网络从序列的开头运行到结尾。这对我们来说很自然，因为它遵循了我们阅读或听语音的方向。然而，对于需要随机访问输入序列的场景来说，同时在两个方向上运行循环计算会更合理。允许在两个方向上进行计算的 RNN 被称为**双向** RNN，可以通过将循环层包装在一个特殊的 `Bidirectional` 层中来创建。\n",
|
||||
"\n",
|
||||
"> **Note**: `Bidirectional` 层会在其内部创建该层的两个副本，并将其中一个副本的 `go_backwards` 属性设置为 `True`，使其沿着序列的相反方向运行。\n",
|
||||
"\n",
|
||||
"无论是单向还是双向的循环网络，都能够捕获序列中的模式，并将其存储到状态向量中或作为输出返回。与卷积网络类似，我们可以在第一个循环层之后再构建另一个循环层，以捕获更高级别的模式，这些模式是由第一个层提取的低级模式构建而成的。这引出了**多层 RNN**的概念，它由两个或更多的循环网络组成，其中前一层的输出作为输入传递给下一层。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"*图片来源于 [这篇精彩的文章](https://towardsdatascience.com/from-a-lstm-cell-to-a-multilayer-lstm-network-with-pytorch-2899eb5696f3)，作者 Fernando López。*\n",
|
||||
"\n",
|
||||
"Keras 使构建这些网络变得非常简单，因为你只需要向模型中添加更多的循环层即可。对于最后一层以外的所有层，我们需要指定 `return_sequences=True` 参数，因为我们需要该层返回所有中间状态，而不仅仅是循环计算的最终状态。\n",
|
||||
"\n",
|
||||
"让我们为分类问题构建一个两层双向 LSTM。\n",
|
||||
"\n",
|
||||
"> **Note** 这段代码运行时间较长，但它提供了迄今为止我们见过的最高准确率。所以也许值得等待并查看结果。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"5044/7500 [===================>..........] - ETA: 2:33 - loss: 0.3709 - acc: 0.8706\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r5045/7500 [===================>..........] - ETA: 2:33 - loss: 0.3709 - acc: 0.8706"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = keras.models.Sequential([\n",
|
||||
" vectorizer,\n",
|
||||
" keras.layers.Embedding(vocab_size, 128, mask_zero=True),\n",
|
||||
" keras.layers.Bidirectional(keras.layers.LSTM(64,return_sequences=True)),\n",
|
||||
" keras.layers.Bidirectional(keras.layers.LSTM(64)), \n",
|
||||
" keras.layers.Dense(4,activation='softmax')\n",
|
||||
"])\n",
|
||||
"\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(batch_size),\n",
|
||||
" validation_data=ds_test.map(tupelize).batch(batch_size))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## RNN用于其他任务\n",
|
||||
"\n",
|
||||
"到目前为止，我们主要关注使用RNN对文本序列进行分类。但它们还能处理许多其他任务，例如文本生成和机器翻译——我们将在下一单元中讨论这些任务。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。原始语言的文档应被视为权威来源。对于关键信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernel_info": {
|
||||
"name": "conda-env-py37_tensorflow-py"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "py37_tensorflow",
|
||||
"language": "python",
|
||||
"name": "conda-env-py37_tensorflow-py"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.9"
|
||||
},
|
||||
"nteract": {
|
||||
"version": "nteract-front-end@1.0.0"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "81351e61f619b432ff51010a4f993194",
|
||||
"translation_date": "2025-08-31T10:47:20+00:00",
|
||||
"source_file": "lessons/5-NLP/16-RNN/RNNTF.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# 作业：笔记本
|
||||
|
||||
使用与本课程相关的笔记本（可以是 PyTorch 或 TensorFlow 版本），使用您自己的数据集重新运行它们，例如来自 Kaggle 的数据集，并注明出处。重写笔记本以突出您自己的发现。尝试使用不同类型的数据集并记录您的发现，例如使用类似 [这个关于天气推文的 Kaggle 比赛数据集](https://www.kaggle.com/competitions/crowdflower-weather-twitter/data?select=train.csv) 的文本。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。因使用本翻译而引起的任何误解或误读，我们概不负责。
|
||||
|
|
@ -0,0 +1,414 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 生成网络\n",
|
||||
"\n",
|
||||
"循环神经网络（RNNs）及其门控单元变体（如长短期记忆单元 LSTMs 和门控循环单元 GRUs）提供了一种语言建模的机制，即它们可以学习单词的排列顺序，并预测序列中下一个单词。这使得我们可以将 RNN 用于**生成任务**，例如普通文本生成、机器翻译，甚至图像描述生成。\n",
|
||||
"\n",
|
||||
"在我们上一单元讨论的 RNN 架构中，每个 RNN 单元会生成下一个隐藏状态作为输出。然而，我们也可以为每个循环单元添加另一个输出，这样就可以输出一个**序列**（长度与原始序列相等）。此外，我们还可以使用不在每一步接受输入的 RNN 单元，而是仅接受一个初始状态向量，然后生成一系列输出。\n",
|
||||
"\n",
|
||||
"在本笔记中，我们将专注于帮助我们生成文本的简单生成模型。为了简化起见，让我们构建一个**字符级网络**，逐字母生成文本。在训练过程中，我们需要获取一些文本语料库，并将其拆分为字母序列。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Loading dataset...\n",
|
||||
"Building vocab...\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"import torchtext\n",
|
||||
"import numpy as np\n",
|
||||
"from torchnlp import *\n",
|
||||
"train_dataset,test_dataset,classes,vocab = load_dataset()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 构建字符词汇表\n",
|
||||
"\n",
|
||||
"为了构建基于字符的生成网络，我们需要将文本拆分为单个字符，而不是单词。这可以通过定义一个不同的分词器来实现：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Vocabulary size = 82\n",
|
||||
"Encoding of 'a' is 1\n",
|
||||
"Character with code 13 is c\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def char_tokenizer(words):\n",
|
||||
" return list(words) #[word for word in words]\n",
|
||||
"\n",
|
||||
"counter = collections.Counter()\n",
|
||||
"for (label, line) in train_dataset:\n",
|
||||
" counter.update(char_tokenizer(line))\n",
|
||||
"vocab = torchtext.vocab.vocab(counter)\n",
|
||||
"\n",
|
||||
"vocab_size = len(vocab)\n",
|
||||
"print(f\"Vocabulary size = {vocab_size}\")\n",
|
||||
"print(f\"Encoding of 'a' is {vocab.get_stoi()['a']}\")\n",
|
||||
"print(f\"Character with code 13 is {vocab.get_itos()[13]}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"让我们看看如何对数据集中的文本进行编码的示例：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"tensor([ 0, 1, 2, 2, 3, 4, 5, 6, 3, 7, 8, 1, 9, 10, 3, 11, 2, 1,\n",
|
||||
" 12, 3, 7, 1, 13, 14, 3, 15, 16, 5, 17, 3, 5, 18, 8, 3, 7, 2,\n",
|
||||
" 1, 13, 14, 3, 19, 20, 8, 21, 5, 8, 9, 10, 22, 3, 20, 8, 21, 5,\n",
|
||||
" 8, 9, 10, 3, 23, 3, 4, 18, 17, 9, 5, 23, 10, 8, 2, 2, 8, 9,\n",
|
||||
" 10, 24, 3, 0, 1, 2, 2, 3, 4, 5, 9, 8, 8, 5, 25, 10, 3, 26,\n",
|
||||
" 12, 27, 16, 26, 2, 27, 16, 28, 29, 30, 1, 16, 26, 3, 17, 31, 3, 21,\n",
|
||||
" 2, 5, 9, 1, 23, 13, 32, 16, 27, 13, 10, 24, 3, 1, 9, 8, 3, 10,\n",
|
||||
" 8, 8, 27, 16, 28, 3, 28, 9, 8, 8, 16, 3, 1, 28, 1, 27, 16, 6])"
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def enc(x):\n",
|
||||
" return torch.LongTensor(encode(x,voc=vocab,tokenizer=char_tokenizer))\n",
|
||||
"\n",
|
||||
"enc(train_dataset[0][1])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 训练生成式 RNN\n",
|
||||
"\n",
|
||||
"我们将通过以下方式训练 RNN 来生成文本。在每一步中，我们会取一个长度为 `nchars` 的字符序列，并让网络为每个输入字符生成下一个输出字符：\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"根据实际场景，我们可能还需要包含一些特殊字符，比如 *序列结束符* `<eos>`。在我们的例子中，我们只是想训练网络进行无尽的文本生成，因此我们会将每个序列的大小固定为 `nchars` 个标记。因此，每个训练样本将由 `nchars` 个输入和 `nchars` 个输出组成（输出是输入序列向左偏移一个符号）。一个小批量（minibatch）将由若干这样的序列组成。\n",
|
||||
"\n",
|
||||
"我们生成小批量的方式是取每条长度为 `l` 的新闻文本，并从中生成所有可能的输入-输出组合（这样的组合会有 `l-nchars` 个）。这些组合将形成一个小批量，而每次训练步骤中小批量的大小会有所不同。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(tensor([[ 0, 1, 2, ..., 28, 29, 30],\n",
|
||||
" [ 1, 2, 2, ..., 29, 30, 1],\n",
|
||||
" [ 2, 2, 3, ..., 30, 1, 16],\n",
|
||||
" ...,\n",
|
||||
" [20, 8, 21, ..., 1, 28, 1],\n",
|
||||
" [ 8, 21, 5, ..., 28, 1, 27],\n",
|
||||
" [21, 5, 8, ..., 1, 27, 16]]),\n",
|
||||
" tensor([[ 1, 2, 2, ..., 29, 30, 1],\n",
|
||||
" [ 2, 2, 3, ..., 30, 1, 16],\n",
|
||||
" [ 2, 3, 4, ..., 1, 16, 26],\n",
|
||||
" ...,\n",
|
||||
" [ 8, 21, 5, ..., 28, 1, 27],\n",
|
||||
" [21, 5, 8, ..., 1, 27, 16],\n",
|
||||
" [ 5, 8, 9, ..., 27, 16, 6]]))"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"nchars = 100\n",
|
||||
"\n",
|
||||
"def get_batch(s,nchars=nchars):\n",
|
||||
" ins = torch.zeros(len(s)-nchars,nchars,dtype=torch.long,device=device)\n",
|
||||
" outs = torch.zeros(len(s)-nchars,nchars,dtype=torch.long,device=device)\n",
|
||||
" for i in range(len(s)-nchars):\n",
|
||||
" ins[i] = enc(s[i:i+nchars])\n",
|
||||
" outs[i] = enc(s[i+1:i+nchars+1])\n",
|
||||
" return ins,outs\n",
|
||||
"\n",
|
||||
"get_batch(train_dataset[0][1])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在我们来定义生成网络。它可以基于我们在上一单元中讨论过的任何循环单元（简单循环单元、LSTM 或 GRU）。在我们的示例中，我们将使用 LSTM。\n",
|
||||
"\n",
|
||||
"由于网络以字符作为输入，并且词汇表的大小相对较小，我们不需要嵌入层，直接使用独热编码的输入即可传递给 LSTM 单元。然而，由于我们传递的是字符编号作为输入，因此在传递给 LSTM 之前需要对它们进行独热编码。这可以通过在 `forward` 过程中调用 `one_hot` 函数来完成。输出编码器将是一个线性层，用于将隐藏状态转换为独热编码的输出。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class LSTMGenerator(torch.nn.Module):\n",
|
||||
" def __init__(self, vocab_size, hidden_dim):\n",
|
||||
" super().__init__()\n",
|
||||
" self.rnn = torch.nn.LSTM(vocab_size,hidden_dim,batch_first=True)\n",
|
||||
" self.fc = torch.nn.Linear(hidden_dim, vocab_size)\n",
|
||||
"\n",
|
||||
" def forward(self, x, s=None):\n",
|
||||
" x = torch.nn.functional.one_hot(x,vocab_size).to(torch.float32)\n",
|
||||
" x,s = self.rnn(x,s)\n",
|
||||
" return self.fc(x),s"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在训练过程中，我们希望能够对生成的文本进行采样。为此，我们将定义一个 `generate` 函数，该函数会从初始字符串 `start` 开始，生成长度为 `size` 的输出字符串。\n",
|
||||
"\n",
|
||||
"其工作原理如下：首先，我们将整个初始字符串 `start` 传递给网络，获取输出状态 `s` 和下一个预测字符 `out`。由于 `out` 是独热编码的，我们通过 `argmax` 获取字符在词汇表中的索引 `nc`，然后使用 `itos` 找到实际字符，并将其添加到结果字符列表 `chars` 中。生成一个字符的这个过程会重复执行 `size` 次，以生成所需数量的字符。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def generate(net,size=100,start='today '):\n",
|
||||
" chars = list(start)\n",
|
||||
" out, s = net(enc(chars).view(1,-1).to(device))\n",
|
||||
" for i in range(size):\n",
|
||||
" nc = torch.argmax(out[0][-1])\n",
|
||||
" chars.append(vocab.get_itos()[nc])\n",
|
||||
" out, s = net(nc.view(1,-1),s)\n",
|
||||
" return ''.join(chars)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在开始训练吧！训练循环与我们之前的所有示例几乎相同，但这次我们每隔1000个epoch打印一次生成的文本样本，而不是打印准确率。\n",
|
||||
"\n",
|
||||
"需要特别注意的是我们计算损失的方式。我们需要根据独热编码的输出 `out` 和预期的文本 `text_out`（字符索引列表）来计算损失。幸运的是，`cross_entropy` 函数的第一个参数需要未归一化的网络输出，第二个参数需要类别编号，这正好符合我们的需求。它还会自动对小批量的大小进行平均。\n",
|
||||
"\n",
|
||||
"我们还通过 `samples_to_train` 限制训练样本数量，以避免等待时间过长。我们鼓励你进行实验，尝试更长时间的训练，可能是多个epoch（在这种情况下，你需要在这段代码外再创建一个循环）。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Current loss = 4.398899078369141\n",
|
||||
"today sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr s\n",
|
||||
"Current loss = 2.161320447921753\n",
|
||||
"today and to the tor to to the tor to to the tor to to the tor to to the tor to to the tor to to the tor t\n",
|
||||
"Current loss = 1.6722588539123535\n",
|
||||
"today and the court to the could to the could to the could to the could to the could to the could to the c\n",
|
||||
"Current loss = 2.423795223236084\n",
|
||||
"today and a second to the conternation of the conternation of the conternation of the conternation of the \n",
|
||||
"Current loss = 1.702607274055481\n",
|
||||
"today and the company to the company to the company to the company to the company to the company to the co\n",
|
||||
"Current loss = 1.692358136177063\n",
|
||||
"today and the company to the company to the company to the company to the company to the company to the co\n",
|
||||
"Current loss = 1.9722288846969604\n",
|
||||
"today and the control the control the control the control the control the control the control the control \n",
|
||||
"Current loss = 1.8705692291259766\n",
|
||||
"today and the second to the second to the second to the second to the second to the second to the second t\n",
|
||||
"Current loss = 1.7626899480819702\n",
|
||||
"today and a security and a security and a security and a security and a security and a security and a secu\n",
|
||||
"Current loss = 1.5574463605880737\n",
|
||||
"today and the company and the company and the company and the company and the company and the company and \n",
|
||||
"Current loss = 1.5620026588439941\n",
|
||||
"today and the be that the be the be that the be the be that the be the be that the be the be that the be t\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"net = LSTMGenerator(vocab_size,64).to(device)\n",
|
||||
"\n",
|
||||
"samples_to_train = 10000\n",
|
||||
"optimizer = torch.optim.Adam(net.parameters(),0.01)\n",
|
||||
"loss_fn = torch.nn.CrossEntropyLoss()\n",
|
||||
"net.train()\n",
|
||||
"for i,x in enumerate(train_dataset):\n",
|
||||
" # x[0] is class label, x[1] is text\n",
|
||||
" if len(x[1])-nchars<10:\n",
|
||||
" continue\n",
|
||||
" samples_to_train-=1\n",
|
||||
" if not samples_to_train: break\n",
|
||||
" text_in, text_out = get_batch(x[1])\n",
|
||||
" optimizer.zero_grad()\n",
|
||||
" out,s = net(text_in)\n",
|
||||
" loss = torch.nn.functional.cross_entropy(out.view(-1,vocab_size),text_out.flatten()) #cross_entropy(out,labels)\n",
|
||||
" loss.backward()\n",
|
||||
" optimizer.step()\n",
|
||||
" if i%1000==0:\n",
|
||||
" print(f\"Current loss = {loss.item()}\")\n",
|
||||
" print(generate(net))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"这个示例已经生成了一些相当不错的文本，但仍有几个方面可以进一步改进：\n",
|
||||
"\n",
|
||||
"* **更好的小批量生成**。我们在训练数据准备时，是从一个样本中生成一个小批量。这种方法并不理想，因为小批量的大小各不相同，有些甚至无法生成，因为文本长度小于 `nchars`。此外，较小的小批量无法充分利用 GPU 的性能。更明智的做法是从所有样本中获取一个较大的文本块，然后生成所有输入-输出对，打乱顺序，并生成大小相等的小批量。\n",
|
||||
"\n",
|
||||
"* **多层 LSTM**。尝试使用 2 或 3 层的 LSTM 单元是有意义的。正如我们在前一个单元中提到的，每一层 LSTM 都会从文本中提取特定的模式。在字符级生成器的情况下，我们可以预期较低的 LSTM 层负责提取音节，而较高的层负责提取单词和单词组合。这可以通过向 LSTM 构造函数传递层数参数来简单实现。\n",
|
||||
"\n",
|
||||
"* 你还可以尝试使用 **GRU 单元**，看看哪种表现更好，以及尝试 **不同的隐藏层大小**。隐藏层过大可能导致过拟合（例如，网络会学习到精确的文本），而隐藏层过小可能无法生成良好的结果。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 软文本生成与温度\n",
|
||||
"\n",
|
||||
"在之前定义的 `generate` 方法中，我们总是选择概率最高的字符作为生成文本中的下一个字符。这导致生成的文本经常在相同的字符序列之间“循环”，如下例所示：\n",
|
||||
"```\n",
|
||||
"today of the second the company and a second the company ...\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"然而，如果我们查看下一个字符的概率分布，可能会发现几个最高概率之间的差异并不大，例如，一个字符的概率可能是 0.2，而另一个是 0.19，等等。例如，当寻找序列 '*play*' 的下一个字符时，下一个字符可能同样可能是空格，或者是 **e**（如单词 *player* 中的情况）。\n",
|
||||
"\n",
|
||||
"这让我们得出一个结论：选择概率最高的字符并不总是“公平”的，因为选择第二高的字符也可能生成有意义的文本。更明智的做法是从网络输出给出的概率分布中**采样**字符。\n",
|
||||
"\n",
|
||||
"这种采样可以通过实现所谓**多项分布**的 `multinomial` 函数来完成。下面定义了一个实现这种**软**文本生成的函数：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"--- Temperature = 0.3\n",
|
||||
"Today and a company and complete an all the land the restrational the as a security and has provers the pay to and a report and the computer in the stand has filities and working the law the stations for a company and with the company and the final the first company and refight of the state and and workin\n",
|
||||
"\n",
|
||||
"--- Temperature = 0.8\n",
|
||||
"Today he oniis its first to Aus bomblaties the marmation a to manan boogot that pirate assaid a relaid their that goverfin the the Cappets Ecrotional Assonia Cition targets it annight the w scyments Blamity #39;s TVeer Diercheg Reserals fran envyuil that of ster said access what succers of Dour-provelith\n",
|
||||
"\n",
|
||||
"--- Temperature = 1.0\n",
|
||||
"Today holy they a 11 will meda a toket subsuaties, engins for Chanos, they's has stainger past to opening orital his thempting new Nattona was al innerforder advan-than #36;s night year his religuled talitatian what the but with Wednesday to Justment will wemen of Mark CCC Camp as Timed Nae wome a leaders\n",
|
||||
"\n",
|
||||
"--- Temperature = 1.3\n",
|
||||
"Today gpone 2.5 fech atcusion poor cocles toparsdorM.cht Line Pamage put 43 his calt lowed to the book, that has authh-the silia rruch ailing to'ory andhes beutirsimi- Aefffive heading offil an auf eacklets is charged evis, Gunymy oy) Mony has it after-sloythyor loveId out filme, the Natabl -Najuntaxiggs \n",
|
||||
"\n",
|
||||
"--- Temperature = 1.8\n",
|
||||
"Today plary, P.slan chly\\401 mardregationly #39;t 8.1Mide) closes ,filtcon alfly playin roven!\\grea.-QFBEP: Iss onfarchQ/itilia CCf Zivesigntwasta orce.-Peul-aw.uicrin of fuglinfsut aftaningwo, MIEX awayew Aice Woiduar Corvagiugge oppo esig ThusBratourid canthly-RyI.co lagitems\\eexciaishes.conBabntusmor I\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def generate_soft(net,size=100,start='today ',temperature=1.0):\n",
|
||||
" chars = list(start)\n",
|
||||
" out, s = net(enc(chars).view(1,-1).to(device))\n",
|
||||
" for i in range(size):\n",
|
||||
" #nc = torch.argmax(out[0][-1])\n",
|
||||
" out_dist = out[0][-1].div(temperature).exp()\n",
|
||||
" nc = torch.multinomial(out_dist,1)[0]\n",
|
||||
" chars.append(vocab.get_itos()[nc])\n",
|
||||
" out, s = net(nc.view(1,-1),s)\n",
|
||||
" return ''.join(chars)\n",
|
||||
" \n",
|
||||
"for i in [0.3,0.8,1.0,1.3,1.8]:\n",
|
||||
" print(f\"--- Temperature = {i}\\n{generate_soft(net,size=300,start='Today ',temperature=i)}\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们引入了一个名为**温度**的参数，用于指示我们应多大程度上坚持最高概率。如果温度为1.0，我们进行公平的多项式采样，而当温度趋于无穷大时——所有概率变得相等，我们随机选择下一个字符。在下面的示例中，我们可以观察到，当我们将温度提高得过高时，文本变得毫无意义，而当温度接近0时，它类似于“循环”的硬生成文本。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.12 ('py38')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "7673cd150d96c74c6d6011460094efb4",
|
||||
"translation_date": "2025-08-31T10:34:37+00:00",
|
||||
"source_file": "lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,497 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 生成网络\n",
|
||||
"\n",
|
||||
"循环神经网络（RNN）及其门控单元变体（如长短期记忆单元 LSTM 和门控循环单元 GRU）提供了一种语言建模的机制，即它们可以学习单词的顺序，并预测序列中下一个单词。这使得我们可以将 RNN 用于**生成任务**，例如普通文本生成、机器翻译，甚至图像描述生成。\n",
|
||||
"\n",
|
||||
"在我们上一单元讨论的 RNN 架构中，每个 RNN 单元会生成下一个隐藏状态作为输出。然而，我们也可以为每个循环单元添加另一个输出，这样就可以输出一个**序列**（其长度等于原始序列的长度）。此外，我们还可以使用不在每一步接受输入的 RNN 单元，而是仅接受一个初始状态向量，然后生成一个输出序列。\n",
|
||||
"\n",
|
||||
"在本笔记中，我们将专注于帮助我们生成文本的简单生成模型。为了简化起见，让我们构建一个**字符级网络**，逐字母生成文本。在训练过程中，我们需要获取一些文本语料库，并将其拆分为字母序列。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorflow import keras\n",
|
||||
"import tensorflow_datasets as tfds\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"ds_train, ds_test = tfds.load('ag_news_subset').values()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 构建字符词汇表\n",
|
||||
"\n",
|
||||
"为了构建字符级生成网络，我们需要将文本拆分为单个字符，而不是单词。我们之前使用的 `TextVectorization` 层无法实现这一点，因此我们有两个选择：\n",
|
||||
"\n",
|
||||
"* 手动加载文本并自行进行分词，就像 [这个官方 Keras 示例](https://keras.io/examples/generative/lstm_character_level_text_generation/) 中所展示的那样\n",
|
||||
"* 使用 `Tokenizer` 类进行字符级分词\n",
|
||||
"\n",
|
||||
"我们将选择第二种方法。`Tokenizer` 也可以用于按单词分词，因此可以很轻松地从字符级分词切换到单词级分词。\n",
|
||||
"\n",
|
||||
"要进行字符级分词，我们需要传递参数 `char_level=True`：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def extract_text(x):\n",
|
||||
" return x['title']+' '+x['description']\n",
|
||||
"\n",
|
||||
"def tupelize(x):\n",
|
||||
" return (extract_text(x),x['label'])\n",
|
||||
"\n",
|
||||
"tokenizer = keras.preprocessing.text.Tokenizer(char_level=True,lower=False)\n",
|
||||
"tokenizer.fit_on_texts([x['title'].numpy().decode('utf-8') for x in ds_train])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们还希望使用一个特殊的标记来表示**序列结束**，我们将其称为`<eos>`。让我们手动将其添加到词汇表中：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"eos_token = len(tokenizer.word_index)+1\n",
|
||||
"tokenizer.word_index['<eos>'] = eos_token\n",
|
||||
"\n",
|
||||
"vocab_size = eos_token + 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在，要将文本编码为数字序列，我们可以使用：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[[48, 2, 10, 10, 5, 44, 1, 25, 5, 8, 10, 13, 78]]"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"tokenizer.texts_to_sequences(['Hello, world!'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 训练生成式 RNN 来生成标题\n",
|
||||
"\n",
|
||||
"我们将通过以下方式训练 RNN 来生成新闻标题。在每一步中，我们会选取一个标题，将其输入到 RNN 中，并让网络根据每个输入字符生成下一个输出字符：\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"对于序列的最后一个字符，我们会让网络生成 `<eos>` 标记。\n",
|
||||
"\n",
|
||||
"我们这里使用的生成式 RNN 的主要区别在于，我们会从 RNN 的每一步输出中获取结果，而不仅仅是从最后一个单元获取。这可以通过为 RNN 单元指定 `return_sequences` 参数来实现。\n",
|
||||
"\n",
|
||||
"因此，在训练过程中，网络的输入将是某个长度的编码字符序列，输出将是相同长度的序列，但会向后偏移一个元素并以 `<eos>` 结束。小批量数据将由多个这样的序列组成，我们需要使用**填充**来对齐所有序列。\n",
|
||||
"\n",
|
||||
"接下来，我们将创建一些函数来转换数据集。由于我们希望在小批量级别对序列进行填充，我们会先通过调用 `.batch()` 对数据集进行分批处理，然后使用 `map` 来进行转换。因此，转换函数将以整个小批量数据作为参数：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def title_batch(x):\n",
|
||||
" x = [t.numpy().decode('utf-8') for t in x]\n",
|
||||
" z = tokenizer.texts_to_sequences(x)\n",
|
||||
" z = tf.keras.preprocessing.sequence.pad_sequences(z)\n",
|
||||
" return tf.one_hot(z,vocab_size), tf.one_hot(tf.concat([z[:,1:],tf.constant(eos_token,shape=(len(z),1))],axis=1),vocab_size)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们在这里做的一些重要事情：\n",
|
||||
"* 我们首先从字符串张量中提取实际文本\n",
|
||||
"* `text_to_sequences` 将字符串列表转换为整数张量列表\n",
|
||||
"* `pad_sequences` 将这些张量填充到它们的最大长度\n",
|
||||
"* 最后，我们对所有字符进行独热编码，同时进行偏移和添加 `<eos>`。我们很快会看到为什么需要独热编码的字符\n",
|
||||
"\n",
|
||||
"然而，这个函数是 **Pythonic** 的，也就是说，它无法自动转换为 Tensorflow 的计算图。如果我们尝试直接在 `Dataset.map` 函数中使用这个函数，会出现错误。我们需要通过使用 `py_function` 包装器来封装这个 Pythonic 调用：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def title_batch_fn(x):\n",
|
||||
" x = x['title']\n",
|
||||
" a,b = tf.py_function(title_batch,inp=[x],Tout=(tf.float32,tf.float32))\n",
|
||||
" return a,b"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> **注意**: 区分 Pythonic 和 Tensorflow 的数据转换函数可能看起来有些复杂，你可能会疑惑为什么我们不在将数据集传递给 `fit` 之前，使用标准的 Python 函数进行转换。虽然这样确实是可行的，但使用 `Dataset.map` 有一个巨大的优势，因为数据转换管道是通过 Tensorflow 的计算图执行的，这可以利用 GPU 的计算能力，并最大限度地减少在 CPU 和 GPU 之间传递数据的需求。\n",
|
||||
"\n",
|
||||
"现在我们可以构建生成器网络并开始训练了。它可以基于我们在上一单元中讨论的任何一种循环单元（简单的、LSTM 或 GRU）。在我们的示例中，我们将使用 LSTM。\n",
|
||||
"\n",
|
||||
"由于网络以字符作为输入，并且词汇表的大小相对较小，我们不需要嵌入层，独热编码的输入可以直接传递到 LSTM 单元中。输出层将是一个 `Dense` 分类器，用于将 LSTM 的输出转换为独热编码的标记编号。\n",
|
||||
"\n",
|
||||
"此外，由于我们处理的是可变长度的序列，我们可以使用 `Masking` 层来创建一个掩码，从而忽略字符串中填充的部分。这并不是严格必要的，因为我们对 `<eos>` 标记之后的内容并不特别感兴趣，但为了熟悉这种层类型的使用，我们还是会用它。`input_shape` 将是 `(None, vocab_size)`，其中 `None` 表示可变长度的序列，而输出形状同样是 `(None, vocab_size)`，正如你可以从 `summary` 中看到的那样：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"sequential\"\n",
|
||||
"_________________________________________________________________\n",
|
||||
"Layer (type) Output Shape Param # \n",
|
||||
"=================================================================\n",
|
||||
"masking (Masking) (None, None, 84) 0 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"lstm (LSTM) (None, None, 128) 109056 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"dense (Dense) (None, None, 84) 10836 \n",
|
||||
"=================================================================\n",
|
||||
"Total params: 119,892\n",
|
||||
"Trainable params: 119,892\n",
|
||||
"Non-trainable params: 0\n",
|
||||
"_________________________________________________________________\n",
|
||||
"15000/15000 [==============================] - 229s 15ms/step - loss: 1.5385\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7fa40c1245e0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = keras.models.Sequential([\n",
|
||||
" keras.layers.Masking(input_shape=(None,vocab_size)),\n",
|
||||
" keras.layers.LSTM(128,return_sequences=True),\n",
|
||||
" keras.layers.Dense(vocab_size,activation='softmax')\n",
|
||||
"])\n",
|
||||
"\n",
|
||||
"model.summary()\n",
|
||||
"model.compile(loss='categorical_crossentropy')\n",
|
||||
"\n",
|
||||
"model.fit(ds_train.batch(8).map(title_batch_fn))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 生成输出\n",
|
||||
"\n",
|
||||
"现在我们已经训练了模型，接下来我们希望使用它生成一些输出。首先，我们需要一种方法来解码由一系列标记数字表示的文本。为此，我们可以使用 `tokenizer.sequences_to_texts` 函数；然而，它在字符级标记化时效果不佳。因此，我们将从标记器中获取一个标记字典（称为 `word_index`），构建一个反向映射，并编写我们自己的解码函数：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"reverse_map = {val:key for key, val in tokenizer.word_index.items()}\n",
|
||||
"\n",
|
||||
"def decode(x):\n",
|
||||
" return ''.join([reverse_map[t] for t in x])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在，我们开始生成。我们将从某个字符串 `start` 开始，将其编码为一个序列 `inp`，然后在每一步调用我们的网络来推断下一个字符。\n",
|
||||
"\n",
|
||||
"网络的输出 `out` 是一个包含 `vocab_size` 元素的向量，表示每个标记的概率。我们可以通过使用 `argmax` 找到最可能的标记编号。然后将这个字符添加到生成的标记列表中，并继续生成。这个生成一个字符的过程会重复 `size` 次，以生成所需数量的字符。如果在生成过程中遇到 `eos_token`，我们会提前终止。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"'Today #39;s lead to strike for the strike for the strike for the strike (AFP)'"
|
||||
]
|
||||
},
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def generate(model,size=100,start='Today '):\n",
|
||||
" inp = tokenizer.texts_to_sequences([start])[0]\n",
|
||||
" chars = inp\n",
|
||||
" for i in range(size):\n",
|
||||
" out = model(tf.expand_dims(tf.one_hot(inp,vocab_size),0))[0][-1]\n",
|
||||
" nc = tf.argmax(out)\n",
|
||||
" if nc==eos_token:\n",
|
||||
" break\n",
|
||||
" chars.append(nc.numpy())\n",
|
||||
" inp = inp+[nc]\n",
|
||||
" return decode(chars)\n",
|
||||
" \n",
|
||||
"generate(model)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 在训练过程中采样输出\n",
|
||||
"\n",
|
||||
"由于我们没有像*准确率*这样的有用指标，判断模型是否在变得更好的唯一方法是通过在训练过程中**采样**生成的字符串。为此，我们将使用**回调函数**，即可以传递给`fit`函数的函数，这些函数会在训练过程中定期被调用。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Epoch 1/3\n",
|
||||
"15000/15000 [==============================] - 226s 15ms/step - loss: 1.2703\n",
|
||||
"Today #39;s a lead in the company for the strike\n",
|
||||
"Epoch 2/3\n",
|
||||
"15000/15000 [==============================] - 227s 15ms/step - loss: 1.2057\n",
|
||||
"Today #39;s the Market Service on Security Start (AP)\n",
|
||||
"Epoch 3/3\n",
|
||||
"15000/15000 [==============================] - 226s 15ms/step - loss: 1.1752\n",
|
||||
"Today #39;s a line on the strike to start for the start\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7fa40c74e3d0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"sampling_callback = keras.callbacks.LambdaCallback(\n",
|
||||
" on_epoch_end = lambda batch, logs: print(generate(model))\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"model.fit(ds_train.batch(8).map(title_batch_fn),callbacks=[sampling_callback],epochs=3)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"这个示例已经生成了一些相当不错的文本，但仍有多种方式可以进一步改进：\n",
|
||||
"\n",
|
||||
"* **更多文本**。我们仅使用了标题来完成任务，但你可能想尝试使用完整的文本。请记住，RNN在处理长序列时表现并不太好，因此可以将它们拆分成较短的句子，或者始终在固定的序列长度（例如，`num_chars`，假设为256）上进行训练。你可以尝试将上面的示例改造成这样的架构，并参考[官方 Keras 教程](https://keras.io/examples/generative/lstm_character_level_text_generation/)来获得灵感。\n",
|
||||
"\n",
|
||||
"* **多层 LSTM**。尝试使用2或3层的LSTM单元是有意义的。正如我们在前一个单元中提到的，每一层LSTM都会从文本中提取某些模式，而在字符级生成器的情况下，我们可以预期较低的LSTM层负责提取音节，而较高的层负责提取单词和词组组合。这可以通过向LSTM构造函数传递层数参数来简单实现。\n",
|
||||
"\n",
|
||||
"* 你可能还想尝试使用**GRU单元**，看看哪种表现更好，以及尝试**不同的隐藏层大小**。隐藏层过大可能导致过拟合（例如，网络会学习到精确的文本），而较小的隐藏层可能无法生成良好的结果。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 软文本生成与温度\n",
|
||||
"\n",
|
||||
"在之前对 `generate` 的定义中，我们总是选择概率最高的字符作为生成文本中的下一个字符。这导致文本经常在相同的字符序列之间反复“循环”，如下例所示：\n",
|
||||
"```\n",
|
||||
"today of the second the company and a second the company ...\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"然而，如果我们查看下一个字符的概率分布，可能会发现几个最高概率之间的差距并不大，例如，一个字符的概率是 0.2，另一个是 0.19，等等。例如，当寻找序列 '*play*' 的下一个字符时，下一个字符可能同样可能是空格，或者是 **e**（如单词 *player* 中）。\n",
|
||||
"\n",
|
||||
"这让我们得出结论，选择概率较高的字符并不总是“公平”的，因为选择第二高的字符也可能生成有意义的文本。更明智的做法是从网络输出给出的概率分布中**采样**字符。\n",
|
||||
"\n",
|
||||
"这种采样可以通过 `np.multinomial` 函数完成，该函数实现了所谓的**多项分布**。下面定义了一个实现这种**软**文本生成的函数：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 33,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"--- Temperature = 0.3\n",
|
||||
"Today #39;s strike #39; to start at the store return\n",
|
||||
"On Sunday PO to Be Data Profit Up (Reuters)\n",
|
||||
"Moscow, SP wins straight to the Microsoft #39;s control of the space start\n",
|
||||
"President olding of the blast start for the strike to pay <b>...</b>\n",
|
||||
"Little red riding hood ficed to the spam countered in European <b>...</b>\n",
|
||||
"\n",
|
||||
"--- Temperature = 0.8\n",
|
||||
"Today countie strikes ryder missile faces food market blut\n",
|
||||
"On Sunday collores lose-toppy of sale of Bullment in <b>...</b>\n",
|
||||
"Moscow, IBM Diffeiting in Afghan Software Hotels (Reuters)\n",
|
||||
"President Ol Luster for Profit Peaced Raised (AP)\n",
|
||||
"Little red riding hood dace on depart talks #39; bank up\n",
|
||||
"\n",
|
||||
"--- Temperature = 1.0\n",
|
||||
"Today wits House buiting debate fixes #39; supervice stake again\n",
|
||||
"On Sunday arling digital poaching In for level\n",
|
||||
"Moscow, DS Up 7, Top Proble Protest Caprey Mamarian Strike\n",
|
||||
"President teps help of roubler stepted lessabul-Dhalitics (AFP)\n",
|
||||
"Little red riding hood signs on cash in Carter-youb\n",
|
||||
"\n",
|
||||
"--- Temperature = 1.3\n",
|
||||
"Today wits flawer ro, pSIA figat's co DroftwavesIs Talo up\n",
|
||||
"On Sunday hround elitwing wint EU Powerburlinetien\n",
|
||||
"Moscow, Bazz #39;s sentries olymen winnelds' next for Olympite Huc?\n",
|
||||
"President lost securitys from power Elections in Smiltrials\n",
|
||||
"Little red riding hood vides profit, exponituity, profitmainalist-at said listers\n",
|
||||
"\n",
|
||||
"--- Temperature = 1.8\n",
|
||||
"Today #39;It: He deat: N.KA Asside\n",
|
||||
"On Sunday i arry Par aldeup patient Wo stele1\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ename": "KeyError",
|
||||
"evalue": "0",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
||||
"\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)",
|
||||
"\u001b[0;32m<ipython-input-33-db32367a0feb>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"\\n--- Temperature = {i}\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mj\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgenerate_soft\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m300\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mwords\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mtemperature\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
|
||||
"\u001b[0;32m<ipython-input-33-db32367a0feb>\u001b[0m in \u001b[0;36mgenerate_soft\u001b[0;34m(model, size, start, temperature)\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0mchars\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0minp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minp\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnc\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchars\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 14\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 15\u001b[0m \u001b[0mwords\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'Today '\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'On Sunday '\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'Moscow, '\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'President '\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'Little red riding hood '\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||||
"\u001b[0;32m<ipython-input-10-3f5fa6130b1d>\u001b[0m in \u001b[0;36mdecode\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mreverse_map\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
|
||||
"\u001b[0;32m<ipython-input-10-3f5fa6130b1d>\u001b[0m in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mreverse_map\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
|
||||
"\u001b[0;31mKeyError\u001b[0m: 0"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def generate_soft(model,size=100,start='Today ',temperature=1.0):\n",
|
||||
" inp = tokenizer.texts_to_sequences([start])[0]\n",
|
||||
" chars = inp\n",
|
||||
" for i in range(size):\n",
|
||||
" out = model(tf.expand_dims(tf.one_hot(inp,vocab_size),0))[0][-1]\n",
|
||||
" probs = tf.exp(tf.math.log(out)/temperature).numpy().astype(np.float64)\n",
|
||||
" probs = probs/np.sum(probs)\n",
|
||||
" nc = np.argmax(np.random.multinomial(1,probs,1))\n",
|
||||
" if nc==eos_token:\n",
|
||||
" break\n",
|
||||
" chars.append(nc)\n",
|
||||
" inp = inp+[nc]\n",
|
||||
" return decode(chars)\n",
|
||||
"\n",
|
||||
"words = ['Today ','On Sunday ','Moscow, ','President ','Little red riding hood ']\n",
|
||||
" \n",
|
||||
"for i in [0.3,0.8,1.0,1.3,1.8]:\n",
|
||||
" print(f\"\\n--- Temperature = {i}\")\n",
|
||||
" for j in range(5):\n",
|
||||
" print(generate_soft(model,size=300,start=words[j],temperature=i))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们引入了一个名为**温度**的参数，用于指示我们应多大程度上坚持最高概率。如果温度为1.0，我们进行公平的多项式采样，而当温度趋于无穷大时——所有概率变得相等，我们随机选择下一个字符。在下面的示例中，我们可以观察到，当我们将温度增加得过高时，文本变得毫无意义，而当温度接近0时，它类似于“循环”的硬生成文本。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.12 ('py38')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "9fbb7d5fda708537649f71f5f646fcde",
|
||||
"translation_date": "2025-08-31T10:32:32+00:00",
|
||||
"source_file": "lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
# 生成网络
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/33)
|
||||
|
||||
循环神经网络（RNN）及其门控单元变体（如长短时记忆单元 LSTM 和门控循环单元 GRU）提供了一种语言建模机制，它们可以学习单词的顺序并预测序列中的下一个单词。这使得我们可以使用 RNN 执行**生成任务**，例如普通文本生成、机器翻译，甚至图像描述。
|
||||
|
||||
> ✅ 想想你在输入时受益于文本补全等生成任务的所有场景。研究一下你喜欢的应用程序，看看它们是否使用了 RNN。
|
||||
|
||||
在我们上一单元讨论的 RNN 架构中，每个 RNN 单元会生成下一个隐藏状态作为输出。然而，我们也可以为每个循环单元添加另一个输出，这样就可以输出一个**序列**（与原始序列长度相等）。此外，我们可以使用不在每一步接受输入的 RNN 单元，只接受一些初始状态向量，然后生成一系列输出。
|
||||
|
||||
这使得可以构建不同的神经网络架构，如下图所示：
|
||||
|
||||
|
||||
|
||||
> 图片来源于 [Andrej Karpaty](http://karpathy.github.io/) 的博客文章 [循环神经网络的非凡有效性](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
|
||||
|
||||
* **一对一**是传统的神经网络，具有一个输入和一个输出
|
||||
* **一对多**是一种生成架构，接受一个输入值并生成一系列输出值。例如，如果我们想训练一个**图像描述**网络来生成图片的文本描述，可以将图片作为输入，通过 CNN 获得其隐藏状态，然后通过循环链逐字生成描述
|
||||
* **多对一**对应于我们在上一单元中描述的 RNN 架构，例如文本分类
|
||||
* **多对多**或**序列到序列**对应于任务如**机器翻译**，其中第一个 RNN 收集输入序列中的所有信息到隐藏状态，另一个 RNN 链将该状态展开为输出序列。
|
||||
|
||||
在本单元中，我们将重点关注帮助生成文本的简单生成模型。为了简化，我们将使用字符级标记化。
|
||||
|
||||
我们将训练这个 RNN 逐步生成文本。在每一步中，我们将取长度为 `nchars` 的字符序列，并要求网络为每个输入字符生成下一个输出字符：
|
||||
|
||||
|
||||
|
||||
在生成文本（推理阶段）时，我们从某个**提示**开始，将其传递给 RNN 单元以生成中间状态，然后从该状态开始生成。我们一次生成一个字符，并将状态和生成的字符传递给另一个 RNN 单元以生成下一个字符，直到生成足够的字符。
|
||||
|
||||
<img src="../../../../../translated_images/zh-CN/rnn-generate-inf.5168dc65e0370eea.webp" width="60%"/>
|
||||
|
||||
> 图片由作者提供
|
||||
|
||||
## ✍️ 练习：生成网络
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
* [使用 PyTorch 的生成网络](GenerativePyTorch.ipynb)
|
||||
* [使用 TensorFlow 的生成网络](GenerativeTF.ipynb)
|
||||
|
||||
## 软文本生成与温度
|
||||
|
||||
每个 RNN 单元的输出是一个字符的概率分布。如果我们总是选择概率最高的字符作为生成文本的下一个字符，文本可能会出现“循环”现象，重复相同的字符序列，如以下示例：
|
||||
|
||||
```
|
||||
today of the second the company and a second the company ...
|
||||
```
|
||||
|
||||
然而，如果我们查看下一个字符的概率分布，可能会发现几个最高概率之间的差异并不大，例如一个字符的概率为 0.2，另一个为 0.19 等。例如，在寻找序列 '*play*' 的下一个字符时，下一个字符可能是空格，也可能是 **e**（如单词 *player*）。
|
||||
|
||||
这让我们得出结论：选择概率最高的字符并不总是“公平”的，因为选择第二高的字符仍可能生成有意义的文本。更明智的做法是从网络输出的概率分布中**采样**字符。我们还可以使用一个参数 **温度**，来平滑概率分布，以增加随机性，或者使分布更陡峭，以更倾向于选择最高概率的字符。
|
||||
|
||||
在上述链接的笔记本中探索软文本生成的实现方式。
|
||||
|
||||
## 总结
|
||||
|
||||
虽然文本生成本身可能很有用，但其主要优势在于能够从某些初始特征向量使用 RNN 生成文本。例如，文本生成可以作为机器翻译的一部分（序列到序列，在这种情况下，来自*编码器*的状态向量用于生成或*解码*翻译后的消息），或者生成图像的文本描述（在这种情况下，特征向量来自 CNN 提取器）。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
在 Microsoft Learn 上学习相关课程：
|
||||
|
||||
* 使用 [PyTorch](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-pytorch/6-generative-networks/?WT.mc_id=academic-77998-cacaste)/[TensorFlow](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-tensorflow/5-generative-networks/?WT.mc_id=academic-77998-cacaste) 进行文本生成
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/34)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
以下文章可以扩展你的知识：
|
||||
|
||||
* 使用马尔可夫链、LSTM 和 GPT-2 进行文本生成的不同方法：[博客文章](https://towardsdatascience.com/text-generation-gpt-2-lstm-markov-chain-9ea371820e1e)
|
||||
* [Keras 文档](https://keras.io/examples/generative/lstm_character_level_text_generation/)中的文本生成示例
|
||||
|
||||
## [作业](lab/README.md)
|
||||
|
||||
我们已经了解了如何逐字符生成文本。在实验中，你将探索逐词生成文本。
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# 使用 RNN 进行词级文本生成
|
||||
|
||||
来自 [AI 初学者课程](https://github.com/microsoft/ai-for-beginners) 的实验任务。
|
||||
|
||||
## 任务
|
||||
|
||||
在本实验中，你需要选择任意一本书，并将其用作数据集来训练一个词级文本生成器。
|
||||
|
||||
## 数据集
|
||||
|
||||
你可以使用任何书籍作为数据集。你可以在 [Project Gutenberg](https://www.gutenberg.org/) 找到许多免费的文本，例如，这里是 Lewis Carroll 的 [爱丽丝梦游仙境](https://www.gutenberg.org/files/11/11-0.txt) 的直接链接。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。原始语言的文档应被视为权威来源。对于关键信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
# 注意机制与Transformer
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/35)
|
||||
|
||||
在自然语言处理领域中，**机器翻译**是一个非常重要的问题，它是像谷歌翻译这样的工具的核心任务。在本节中，我们将重点讨论机器翻译，或者更广泛地说，任何*序列到序列*任务（也称为**句子转换**）。
|
||||
|
||||
使用RNN时，序列到序列任务通过两个循环网络实现，其中一个网络，即**编码器**，将输入序列压缩为一个隐藏状态，而另一个网络，即**解码器**，将隐藏状态展开为翻译结果。这种方法存在一些问题：
|
||||
|
||||
* 编码器网络的最终状态难以记住句子的开头，从而导致模型在处理长句子时质量较差。
|
||||
* 序列中的所有词对结果的影响相同。然而在实际中，输入序列中的某些词对输出序列的影响往往比其他词更大。
|
||||
|
||||
**注意机制**提供了一种方法，可以对每个输入向量对RNN输出预测的上下文影响进行加权。其实现方式是创建输入RNN的中间状态与输出RNN之间的快捷路径。这样，在生成输出符号y<sub>t</sub>时，我们会考虑所有输入隐藏状态h<sub>i</sub>，并赋予不同的权重系数α<sub>t,i</sub>。
|
||||
|
||||
|
||||
|
||||
> [Bahdanau等人，2015](https://arxiv.org/pdf/1409.0473.pdf)中的加性注意机制编码器-解码器模型，图片来源于[这篇博客](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html)
|
||||
|
||||
注意矩阵{α<sub>i,j</sub>}表示某些输入词在生成输出序列中某个词时的影响程度。下面是一个这样的矩阵示例：
|
||||
|
||||
|
||||
|
||||
> 图片来自[Bahdanau等人，2015](https://arxiv.org/pdf/1409.0473.pdf)（图3）
|
||||
|
||||
注意机制是当前或接近当前自然语言处理领域的最先进技术的关键所在。然而，添加注意机制会显著增加模型参数的数量，从而导致RNN的扩展问题。RNN扩展的一个关键限制是模型的循环性质使得训练难以批量化和并行化。在RNN中，序列的每个元素都需要按顺序处理，这意味着它无法轻松并行化。
|
||||
|
||||
|
||||
|
||||
> 图片来源于[谷歌博客](https://research.googleblog.com/2016/09/a-neural-network-for-machine.html)
|
||||
|
||||
注意机制的采用结合了这一限制，促成了如今我们所熟知和使用的最先进的Transformer模型的诞生，例如BERT和Open-GPT3。
|
||||
|
||||
## Transformer模型
|
||||
|
||||
Transformer的核心思想之一是避免RNN的顺序处理特性，并创建一个在训练过程中可并行化的模型。这通过以下两个想法实现：
|
||||
|
||||
* 位置编码
|
||||
* 使用自注意机制捕捉模式，而不是使用RNN（或CNN）（这也是为什么介绍Transformer的论文被称为*[Attention is all you need](https://arxiv.org/abs/1706.03762)*）
|
||||
|
||||
### 位置编码/嵌入
|
||||
|
||||
位置编码的思想如下：
|
||||
1. 使用RNN时，标记的相对位置由步数表示，因此不需要显式表示。
|
||||
2. 然而，一旦我们切换到注意机制，我们需要知道序列中标记的相对位置。
|
||||
3. 为了获得位置编码，我们将标记序列与序列中的标记位置序列（即数字序列0,1, ...）结合起来。
|
||||
4. 然后我们将标记位置与标记嵌入向量混合。为了将位置（整数）转换为向量，我们可以使用不同的方法：
|
||||
|
||||
* 可训练嵌入，类似于标记嵌入。这是我们在这里考虑的方法。我们在标记和它们的位置上应用嵌入层，生成维度相同的嵌入向量，然后将它们相加。
|
||||
* 固定位置编码函数，如原始论文中提出的。
|
||||
|
||||
<img src="../../../../../translated_images/zh-CN/pos-embedding.e41ce9b6cf6078af.webp" width="50%"/>
|
||||
|
||||
> 图片由作者提供
|
||||
|
||||
通过位置嵌入，我们可以同时嵌入原始标记及其在序列中的位置。
|
||||
|
||||
### 多头自注意机制
|
||||
|
||||
接下来，我们需要捕捉序列中的一些模式。为此，Transformer使用了**自注意机制**，即将注意机制同时应用于输入和输出的同一序列。应用自注意机制使我们能够考虑句子中的**上下文**，并查看哪些词是相互关联的。例如，它可以帮助我们识别代词*it*所指代的词，并考虑上下文：
|
||||
|
||||
|
||||
|
||||
> 图片来源于[谷歌博客](https://research.googleblog.com/2017/08/transformer-novel-neural-network.html)
|
||||
|
||||
在Transformer中，我们使用**多头注意机制**，以赋予网络捕捉多种依赖关系的能力，例如长期与短期词关系、共指关系与其他关系等。
|
||||
|
||||
[TensorFlow Notebook](TransformersTF.ipynb)中包含有关Transformer层实现的更多细节。
|
||||
|
||||
### 编码器-解码器注意机制
|
||||
|
||||
在Transformer中，注意机制用于两个地方：
|
||||
|
||||
* 使用自注意机制捕捉输入文本中的模式
|
||||
* 执行序列翻译——这是编码器和解码器之间的注意层。
|
||||
|
||||
编码器-解码器注意机制与RNN中使用的注意机制非常相似，如本节开头所述。以下动画图解释了编码器-解码器注意机制的作用。
|
||||
|
||||
|
||||
|
||||
由于每个输入位置独立映射到每个输出位置，Transformer比RNN更容易并行化，这使得能够构建更大、更具表现力的语言模型。每个注意头可以用于学习词之间的不同关系，从而改善下游自然语言处理任务。
|
||||
|
||||
## BERT
|
||||
|
||||
**BERT**（Bidirectional Encoder Representations from Transformers，双向编码器表示）是一个非常大的多层Transformer网络，*BERT-base*有12层，*BERT-large*有24层。模型首先在大规模文本数据（维基百科+书籍）上进行无监督训练（预测句子中的被遮蔽词）。在预训练过程中，模型吸收了大量的语言理解能力，这些能力可以通过微调其他数据集来利用。这一过程称为**迁移学习**。
|
||||
|
||||
|
||||
|
||||
> 图片[来源](http://jalammar.github.io/illustrated-bert/)
|
||||
|
||||
## ✍️ 练习：Transformer
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
* [PyTorch中的Transformer](TransformersPyTorch.ipynb)
|
||||
* [TensorFlow中的Transformer](TransformersTF.ipynb)
|
||||
|
||||
## 总结
|
||||
|
||||
在本课中，你学习了Transformer和注意机制，它们是自然语言处理工具箱中的重要工具。Transformer架构有许多变体，包括BERT、DistilBERT、BigBird、OpenGPT3等，这些模型可以进行微调。[HuggingFace包](https://github.com/huggingface/)提供了一个库，可以使用PyTorch和TensorFlow训练许多这些架构。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/36)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
* [博客文章](https://mchromiak.github.io/articles/2017/Sep/12/Transformer-Attention-is-all-you-need/)，解释了经典的[Attention is all you need](https://arxiv.org/abs/1706.03762)论文。
|
||||
* [一系列博客文章](https://towardsdatascience.com/transformers-explained-visually-part-1-overview-of-functionality-95a6dd460452)，详细解释了Transformer的架构。
|
||||
|
||||
## [作业](assignment.md)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,353 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 注意力机制与Transformer\n",
|
||||
"\n",
|
||||
"循环神经网络的一个主要缺点是序列中的所有单词对结果的影响是相同的。这会导致标准的LSTM编码器-解码器模型在处理序列到序列任务（如命名实体识别和机器翻译）时表现不佳。实际上，输入序列中的某些特定单词往往对输出序列的影响更大。\n",
|
||||
"\n",
|
||||
"以序列到序列模型为例，比如机器翻译。它由两个循环神经网络实现，其中一个网络（**编码器**）将输入序列压缩成隐藏状态，另一个网络（**解码器**）将隐藏状态展开为翻译结果。这种方法的问题在于，网络的最终状态很难记住句子的开头部分，从而导致模型在处理长句子时质量较差。\n",
|
||||
"\n",
|
||||
"**注意力机制**提供了一种方法，可以为每个输入向量对RNN每个输出预测的上下文影响赋予不同的权重。其实现方式是通过在输入RNN的中间状态和输出RNN之间创建快捷连接。这样，在生成输出符号$y_t$时，我们会考虑所有输入隐藏状态$h_i$，并赋予不同的权重系数$\\alpha_{t,i}$。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"*[Bahdanau等人，2015](https://arxiv.org/pdf/1409.0473.pdf)中的加性注意力机制编码器-解码器模型，图片引自[这篇博客](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html)*\n",
|
||||
"\n",
|
||||
"注意力矩阵$\\{\\alpha_{i,j}\\}$表示某些输入单词在生成输出序列中某个单词时的作用程度。以下是这样一个矩阵的示例：\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"*图片取自[Bahdanau等人，2015](https://arxiv.org/pdf/1409.0473.pdf)（图3）*\n",
|
||||
"\n",
|
||||
"注意力机制是当前或接近当前自然语言处理领域最先进技术的核心原因之一。然而，添加注意力机制会显著增加模型参数的数量，这导致了RNN的扩展问题。RNN扩展的一个关键限制是模型的循环特性使得批量处理和并行化训练变得具有挑战性。在RNN中，序列的每个元素都需要按顺序处理，这意味着它无法轻松并行化。\n",
|
||||
"\n",
|
||||
"注意力机制的采用与这一限制相结合，促成了我们今天所熟知和使用的最先进的Transformer模型的诞生，从BERT到OpenGPT3。\n",
|
||||
"\n",
|
||||
"## Transformer模型\n",
|
||||
"\n",
|
||||
"与将每次预测的上下文传递到下一步评估不同，**Transformer模型**使用**位置编码**和注意力机制，在给定的文本窗口内捕获输入的上下文。下图展示了位置编码与注意力机制如何在给定窗口内捕获上下文。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"由于每个输入位置可以独立映射到每个输出位置，Transformer比RNN更容易并行化，这使得构建更大、更具表现力的语言模型成为可能。每个注意力头可以用来学习单词之间的不同关系，从而改进下游的自然语言处理任务。\n",
|
||||
"\n",
|
||||
"**BERT**（基于Transformer的双向编码器表示）是一个非常大的多层Transformer网络，*BERT-base*有12层，*BERT-large*有24层。该模型首先在大规模文本数据（维基百科+书籍）上进行无监督预训练（预测句子中的被遮掩单词）。在预训练过程中，模型吸收了大量的语言理解能力，这些能力可以通过微调其他数据集来利用。这一过程被称为**迁移学习**。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Transformer架构有许多变体，包括BERT、DistilBERT、BigBird、OpenGPT3等，这些模型都可以进行微调。[HuggingFace包](https://github.com/huggingface/)提供了一个用于训练许多这些架构的PyTorch库。\n",
|
||||
"\n",
|
||||
"## 使用BERT进行文本分类\n",
|
||||
"\n",
|
||||
"让我们看看如何使用预训练的BERT模型来解决传统任务：序列分类。我们将对原始的AG News数据集进行分类。\n",
|
||||
"\n",
|
||||
"首先，加载HuggingFace库和我们的数据集：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Loading dataset...\n",
|
||||
"Building vocab...\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"import torchtext\n",
|
||||
"from torchnlp import *\n",
|
||||
"import transformers\n",
|
||||
"train_dataset, test_dataset, classes, vocab = load_dataset()\n",
|
||||
"vocab_len = len(vocab)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"由于我们将使用预训练的 BERT 模型，因此需要使用特定的分词器。首先，我们将加载与预训练 BERT 模型相关联的分词器。\n",
|
||||
"\n",
|
||||
"HuggingFace 库包含一个预训练模型的仓库，您只需在 `from_pretrained` 函数中指定模型名称作为参数即可使用。这些模型所需的所有二进制文件会自动下载。\n",
|
||||
"\n",
|
||||
"然而，在某些情况下，您可能需要加载自己的模型。在这种情况下，您可以指定包含所有相关文件的目录，包括分词器的参数、模型参数的 `config.json` 文件、二进制权重等。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# To load the model from Internet repository using model name. \n",
|
||||
"# Use this if you are running from your own copy of the notebooks\n",
|
||||
"bert_model = 'bert-base-uncased' \n",
|
||||
"\n",
|
||||
"# To load the model from the directory on disk. Use this for Microsoft Learn module, because we have\n",
|
||||
"# prepared all required files for you.\n",
|
||||
"bert_model = './bert'\n",
|
||||
"\n",
|
||||
"tokenizer = transformers.BertTokenizer.from_pretrained(bert_model)\n",
|
||||
"\n",
|
||||
"MAX_SEQ_LEN = 128\n",
|
||||
"PAD_INDEX = tokenizer.convert_tokens_to_ids(tokenizer.pad_token)\n",
|
||||
"UNK_INDEX = tokenizer.convert_tokens_to_ids(tokenizer.unk_token)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"`tokenizer` 对象包含 `encode` 函数，可直接用于编码文本：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[101, 1052, 22123, 2953, 2818, 2003, 1037, 2307, 7705, 2005, 17953, 2361, 102]"
|
||||
]
|
||||
},
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"tokenizer.encode('PyTorch is a great framework for NLP')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"然后，让我们创建在训练期间用于访问数据的迭代器。由于BERT使用其自己的编码函数，我们需要定义一个类似于之前定义的`padify`的填充函数：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def pad_bert(b):\n",
|
||||
" # b is the list of tuples of length batch_size\n",
|
||||
" # - first element of a tuple = label, \n",
|
||||
" # - second = feature (text sequence)\n",
|
||||
" # build vectorized sequence\n",
|
||||
" v = [tokenizer.encode(x[1]) for x in b]\n",
|
||||
" # compute max length of a sequence in this minibatch\n",
|
||||
" l = max(map(len,v))\n",
|
||||
" return ( # tuple of two tensors - labels and features\n",
|
||||
" torch.LongTensor([t[0] for t in b]),\n",
|
||||
" torch.stack([torch.nn.functional.pad(torch.tensor(t),(0,l-len(t)),mode='constant',value=0) for t in v])\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=8, collate_fn=pad_bert, shuffle=True)\n",
|
||||
"test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=8, collate_fn=pad_bert)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在我们的案例中，我们将使用名为`bert-base-uncased`的预训练BERT模型。让我们使用`BertForSequenceClassification`包加载模型。这确保了我们的模型已经具备分类所需的架构，包括最终的分类器。您会看到一条警告消息，指出最终分类器的权重尚未初始化，模型需要进行预训练——这完全没问题，因为这正是我们即将要做的！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Some weights of the model checkpoint at ./bert were not used when initializing BertForSequenceClassification: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']\n",
|
||||
"- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
|
||||
"- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
|
||||
"Some weights of BertForSequenceClassification were not initialized from the model checkpoint at ./bert and are newly initialized: ['classifier.weight', 'classifier.bias']\n",
|
||||
"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = transformers.BertForSequenceClassification.from_pretrained(bert_model,num_labels=4).to(device)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在我们可以开始训练了！由于 BERT 已经经过预训练，我们希望使用较小的学习率开始训练，以免破坏初始权重。\n",
|
||||
"\n",
|
||||
"所有的核心工作都由 `BertForSequenceClassification` 模型完成。当我们将训练数据传入模型时，它会返回输入小批量数据的损失值和网络输出。我们使用损失值进行参数优化（`loss.backward()` 执行反向传播），而使用 `out` 通过比较计算得到的标签 `labs`（使用 `argmax` 计算）与期望的 `labels` 来计算训练准确率。\n",
|
||||
"\n",
|
||||
"为了控制训练过程，我们会在若干次迭代中累积损失值和准确率，并在每 `report_freq` 个训练周期后打印它们。\n",
|
||||
"\n",
|
||||
"这次训练可能会花费相当长的时间，因此我们会限制迭代次数。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Loss = 1.1254194641113282, Accuracy = 0.585\n",
|
||||
"Loss = 0.6194715118408203, Accuracy = 0.83\n",
|
||||
"Loss = 0.46665248870849607, Accuracy = 0.8475\n",
|
||||
"Loss = 0.4309701919555664, Accuracy = 0.8575\n",
|
||||
"Loss = 0.35427074432373046, Accuracy = 0.8825\n",
|
||||
"Loss = 0.3306886291503906, Accuracy = 0.8975\n",
|
||||
"Loss = 0.30340143203735354, Accuracy = 0.8975\n",
|
||||
"Loss = 0.26139299392700194, Accuracy = 0.915\n",
|
||||
"Loss = 0.26708646774291994, Accuracy = 0.9225\n",
|
||||
"Loss = 0.3667240524291992, Accuracy = 0.8675\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"optimizer = torch.optim.Adam(model.parameters(), lr=2e-5)\n",
|
||||
"\n",
|
||||
"report_freq = 50\n",
|
||||
"iterations = 500 # make this larger to train for longer time!\n",
|
||||
"\n",
|
||||
"model.train()\n",
|
||||
"\n",
|
||||
"i,c = 0,0\n",
|
||||
"acc_loss = 0\n",
|
||||
"acc_acc = 0\n",
|
||||
"\n",
|
||||
"for labels,texts in train_loader:\n",
|
||||
" labels = labels.to(device)-1 # get labels in the range 0-3 \n",
|
||||
" texts = texts.to(device)\n",
|
||||
" loss, out = model(texts, labels=labels)[:2]\n",
|
||||
" labs = out.argmax(dim=1)\n",
|
||||
" acc = torch.mean((labs==labels).type(torch.float32))\n",
|
||||
" optimizer.zero_grad()\n",
|
||||
" loss.backward()\n",
|
||||
" optimizer.step()\n",
|
||||
" acc_loss += loss\n",
|
||||
" acc_acc += acc\n",
|
||||
" i+=1\n",
|
||||
" c+=1\n",
|
||||
" if i%report_freq==0:\n",
|
||||
" print(f\"Loss = {acc_loss.item()/c}, Accuracy = {acc_acc.item()/c}\")\n",
|
||||
" c = 0\n",
|
||||
" acc_loss = 0\n",
|
||||
" acc_acc = 0\n",
|
||||
" iterations-=1\n",
|
||||
" if not iterations:\n",
|
||||
" break"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"你可以看到（特别是当你增加迭代次数并等待足够长的时间），BERT分类能够提供相当不错的准确率！这是因为BERT已经很好地理解了语言的结构，我们只需要微调最终的分类器。然而，由于BERT是一个大型模型，整个训练过程耗时较长，并且需要强大的计算能力！（GPU，最好是多块）。\n",
|
||||
"\n",
|
||||
"> **Note:** 在我们的示例中，我们使用的是最小的预训练BERT模型之一。还有更大的模型可能会带来更好的结果。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 评估模型性能\n",
|
||||
"\n",
|
||||
"现在我们可以在测试数据集上评估模型的性能了。评估循环与训练循环非常相似，但我们不能忘记通过调用 `model.eval()` 将模型切换到评估模式。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Final accuracy: 0.9047029702970297\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.eval()\n",
|
||||
"iterations = 100\n",
|
||||
"acc = 0\n",
|
||||
"i = 0\n",
|
||||
"for labels,texts in test_loader:\n",
|
||||
" labels = labels.to(device)-1 \n",
|
||||
" texts = texts.to(device)\n",
|
||||
" _, out = model(texts, labels=labels)[:2]\n",
|
||||
" labs = out.argmax(dim=1)\n",
|
||||
" acc += torch.mean((labs==labels).type(torch.float32))\n",
|
||||
" i+=1\n",
|
||||
" if i>iterations: break\n",
|
||||
" \n",
|
||||
"print(f\"Final accuracy: {acc.item()/i}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 关键点\n",
|
||||
"\n",
|
||||
"在本单元中，我们了解到，从 **transformers** 库中获取预训练语言模型并将其适配到我们的文本分类任务是多么简单。同样，BERT 模型也可以用于实体抽取、问答以及其他 NLP 任务。\n",
|
||||
"\n",
|
||||
"Transformer 模型代表了当前 NLP 的最先进水平，在大多数情况下，它应该是你在实现自定义 NLP 解决方案时首先尝试的选项。然而，如果你想构建高级的神经网络模型，理解本模块中讨论的循环神经网络的基本原理是非常重要的。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "py37_pytorch",
|
||||
"language": "python",
|
||||
"name": "conda-env-py37_pytorch-py"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.7"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "753865967678a92dbce7d7efbd36d980",
|
||||
"translation_date": "2025-08-31T10:39:46+00:00",
|
||||
"source_file": "lessons/5-NLP/18-Transformers/TransformersPyTorch.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,819 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 注意力机制与Transformer\n",
|
||||
"\n",
|
||||
"循环网络的一个主要缺点是序列中的所有单词对结果的影响相同。这会导致标准的LSTM编码器-解码器模型在处理序列到序列任务（如命名实体识别和机器翻译）时表现不佳。实际上，输入序列中的某些特定单词对输出序列的影响往往比其他单词更大。\n",
|
||||
"\n",
|
||||
"以序列到序列模型为例，比如机器翻译。它通过两个循环网络实现，其中一个网络（**编码器**）将输入序列压缩成隐藏状态，另一个网络（**解码器**）将隐藏状态展开成翻译结果。这种方法的问题在于，网络的最终状态很难记住句子的开头部分，从而导致模型在处理长句子时质量较差。\n",
|
||||
"\n",
|
||||
"**注意力机制**提供了一种方法，可以对每个输入向量对RNN输出预测的上下文影响进行加权。它的实现方式是通过在输入RNN的中间状态和输出RNN之间创建快捷路径。这样，在生成输出符号$y_t$时，我们会考虑所有输入隐藏状态$h_i$，并赋予不同的权重系数$\\alpha_{t,i}$。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"*[Bahdanau等人，2015](https://arxiv.org/pdf/1409.0473.pdf)中的编码器-解码器模型，图片引用自[这篇博客](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html)*\n",
|
||||
"\n",
|
||||
"注意力矩阵$\\{\\alpha_{i,j}\\}$表示某些输入单词在生成输出序列中某个单词时的影响程度。下面是一个这样的矩阵示例：\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"*图片取自[Bahdanau等人，2015](https://arxiv.org/pdf/1409.0473.pdf)（图3）*\n",
|
||||
"\n",
|
||||
"注意力机制是当前或接近当前自然语言处理领域的先进技术的核心。然而，添加注意力机制会显著增加模型参数的数量，这导致了循环网络的扩展问题。循环网络扩展的一个关键限制是模型的循环特性使得批量处理和训练并行化变得困难。在循环网络中，序列的每个元素都需要按顺序处理，这意味着它无法轻松并行化。\n",
|
||||
"\n",
|
||||
"注意力机制的采用结合这一限制，促成了如今我们所熟知和使用的先进Transformer模型的诞生，从BERT到OpenGPT3。\n",
|
||||
"\n",
|
||||
"## Transformer模型\n",
|
||||
"\n",
|
||||
"与将每次预测的上下文传递到下一步评估不同，**Transformer模型**使用**位置编码**和**注意力机制**来捕获给定输入在提供的文本窗口中的上下文。下图展示了位置编码与注意力机制如何在给定窗口中捕获上下文。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"由于每个输入位置可以独立映射到每个输出位置，Transformer比循环网络更容易并行化，这使得构建更大、更具表现力的语言模型成为可能。每个注意力头可以用来学习单词之间的不同关系，从而提升下游自然语言处理任务的效果。\n",
|
||||
"\n",
|
||||
"## 构建简单的Transformer模型\n",
|
||||
"\n",
|
||||
"Keras中没有内置的Transformer层，但我们可以自己构建。和之前一样，我们将专注于AG News数据集的文本分类，但值得一提的是，Transformer模型在更复杂的自然语言处理任务中表现最佳。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorflow import keras\n",
|
||||
"import tensorflow_datasets as tfds\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"ds_train, ds_test = tfds.load('ag_news_subset').values()\n",
|
||||
"\n",
|
||||
"def extract_text(x):\n",
|
||||
" return x['title']+' '+x['description']\n",
|
||||
"\n",
|
||||
"def tupelize(x):\n",
|
||||
" return (extract_text(x),x['label'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在 Keras 中，新层应继承 `Layer` 类，并实现 `call` 方法。让我们从 **位置嵌入** 层开始。我们将使用[官方 Keras 文档中的一些代码](https://keras.io/examples/nlp/text_classification_with_transformer/)。我们将假设我们将所有输入序列填充到长度 `maxlen`。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class TokenAndPositionEmbedding(keras.layers.Layer):\n",
|
||||
" def __init__(self, maxlen, vocab_size, embed_dim):\n",
|
||||
" super(TokenAndPositionEmbedding, self).__init__()\n",
|
||||
" self.token_emb = keras.layers.Embedding(input_dim=vocab_size, output_dim=embed_dim)\n",
|
||||
" self.pos_emb = keras.layers.Embedding(input_dim=maxlen, output_dim=embed_dim)\n",
|
||||
" self.maxlen = maxlen\n",
|
||||
"\n",
|
||||
" def call(self, x):\n",
|
||||
" maxlen = self.maxlen\n",
|
||||
" positions = tf.range(start=0, limit=maxlen, delta=1)\n",
|
||||
" positions = self.pos_emb(positions)\n",
|
||||
" x = self.token_emb(x)\n",
|
||||
" return x+positions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"这一层由两个`Embedding`层组成：一个用于嵌入标记（以我们之前讨论的方式），另一个用于嵌入标记位置。标记位置是通过使用`tf.range`从0到`maxlen`生成的一系列自然数创建的，然后通过嵌入层处理。两个生成的嵌入向量随后相加，生成输入的基于位置嵌入的表示，其形状为`maxlen`$\\times$`embed_dim`。\n",
|
||||
"\n",
|
||||
"现在，让我们实现Transformer块。它将接收之前定义的嵌入层的输出：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class TransformerBlock(keras.layers.Layer):\n",
|
||||
" def __init__(self, embed_dim, num_heads, ff_dim, rate=0.1):\n",
|
||||
" super(TransformerBlock, self).__init__()\n",
|
||||
" self.att = keras.layers.MultiHeadAttention(num_heads=num_heads, key_dim=embed_dim, name='attn')\n",
|
||||
" self.ffn = keras.Sequential(\n",
|
||||
" [keras.layers.Dense(ff_dim, activation=\"relu\"), keras.layers.Dense(embed_dim),]\n",
|
||||
" )\n",
|
||||
" self.layernorm1 = keras.layers.LayerNormalization(epsilon=1e-6)\n",
|
||||
" self.layernorm2 = keras.layers.LayerNormalization(epsilon=1e-6)\n",
|
||||
" self.dropout1 = keras.layers.Dropout(rate)\n",
|
||||
" self.dropout2 = keras.layers.Dropout(rate)\n",
|
||||
"\n",
|
||||
" def call(self, inputs, training):\n",
|
||||
" attn_output = self.att(inputs, inputs)\n",
|
||||
" attn_output = self.dropout1(attn_output, training=training)\n",
|
||||
" out1 = self.layernorm1(inputs + attn_output)\n",
|
||||
" ffn_output = self.ffn(out1)\n",
|
||||
" ffn_output = self.dropout2(ffn_output, training=training)\n",
|
||||
" return self.layernorm2(out1 + ffn_output)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在，我们已经准备好定义完整的 Transformer 模型：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"sequential_1\"\n",
|
||||
"_________________________________________________________________\n",
|
||||
"Layer (type) Output Shape Param # \n",
|
||||
"=================================================================\n",
|
||||
"text_vectorization (TextVect (None, 256) 0 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"token_and_position_embedding (None, 256, 32) 648192 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"transformer_block (Transform (None, 256, 32) 10656 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"global_average_pooling1d (Gl (None, 32) 0 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"dropout_2 (Dropout) (None, 32) 0 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"dense_2 (Dense) (None, 20) 660 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"dropout_3 (Dropout) (None, 20) 0 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"dense_3 (Dense) (None, 4) 84 \n",
|
||||
"=================================================================\n",
|
||||
"Total params: 659,592\n",
|
||||
"Trainable params: 659,592\n",
|
||||
"Non-trainable params: 0\n",
|
||||
"_________________________________________________________________\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"embed_dim = 32 # Embedding size for each token\n",
|
||||
"num_heads = 2 # Number of attention heads\n",
|
||||
"ff_dim = 32 # Hidden layer size in feed forward network inside transformer\n",
|
||||
"maxlen = 256\n",
|
||||
"vocab_size = 20000\n",
|
||||
"\n",
|
||||
"model = keras.models.Sequential([\n",
|
||||
" keras.layers.experimental.preprocessing.TextVectorization(max_tokens=vocab_size,output_sequence_length=maxlen, input_shape=(1,)),\n",
|
||||
" TokenAndPositionEmbedding(maxlen, vocab_size, embed_dim),\n",
|
||||
" TransformerBlock(embed_dim, num_heads, ff_dim),\n",
|
||||
" keras.layers.GlobalAveragePooling1D(),\n",
|
||||
" keras.layers.Dropout(0.1),\n",
|
||||
" keras.layers.Dense(20, activation=\"relu\"),\n",
|
||||
" keras.layers.Dropout(0.1),\n",
|
||||
" keras.layers.Dense(4, activation=\"softmax\")\n",
|
||||
"])\n",
|
||||
"\n",
|
||||
"model.summary()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training tokenizer\n",
|
||||
"938/938 [==============================] - 45s 39ms/step - loss: 0.4978 - acc: 0.8068 - val_loss: 0.2808 - val_acc: 0.9124\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7f9c2427a0d0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print('Training tokenizer')\n",
|
||||
"model.layers[0].adapt(ds_train.map(extract_text))\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(128),validation_data=ds_test.map(tupelize).batch(128))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## BERT Transformer 模型\n",
|
||||
"\n",
|
||||
"**BERT**（双向编码器表示的变换器）是一个非常大的多层变换器网络，*BERT-base* 有 12 层，*BERT-large* 有 24 层。该模型首先在大规模文本数据（维基百科 + 书籍）上进行无监督训练（预测句子中被遮盖的单词）。在预训练过程中，模型吸收了大量的语言理解能力，随后可以通过微调与其他数据集结合使用。这一过程被称为 **迁移学习**。\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"变换器架构有许多变体，包括 BERT、DistilBERT、BigBird、OpenGPT3 等，这些都可以进行微调。\n",
|
||||
"\n",
|
||||
"接下来，我们来看如何使用预训练的 BERT 模型来解决传统的序列分类问题。我们将借用 [官方文档](https://www.tensorflow.org/text/tutorials/classify_text_with_bert) 中的一些思路和代码。\n",
|
||||
"\n",
|
||||
"为了加载预训练模型，我们将使用 **Tensorflow hub**。首先，让我们加载 BERT 专用的向量化工具：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "ModuleNotFoundError",
|
||||
"evalue": "No module named 'tensorflow_text'",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
|
||||
"\u001b[1;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)",
|
||||
"\u001b[1;32m~\\AppData\\Local\\Temp/ipykernel_41180/4216669875.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[1;32mimport\u001b[0m \u001b[0mtensorflow_text\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 2\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mtensorflow_hub\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mhub\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 3\u001b[0m \u001b[0mvectorizer\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mhub\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mKerasLayer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
|
||||
"\u001b[1;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow_text'"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import tensorflow_text \n",
|
||||
"import tensorflow_hub as hub\n",
|
||||
"vectorizer = hub.KerasLayer('https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'input_type_ids': <tf.Tensor: shape=(1, 128), dtype=int32, numpy=\n",
|
||||
" array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],\n",
|
||||
" dtype=int32)>,\n",
|
||||
" 'input_word_ids': <tf.Tensor: shape=(1, 128), dtype=int32, numpy=\n",
|
||||
" array([[ 101, 1045, 2293, 19081, 102, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0]], dtype=int32)>,\n",
|
||||
" 'input_mask': <tf.Tensor: shape=(1, 128), dtype=int32, numpy=\n",
|
||||
" array([[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
|
||||
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],\n",
|
||||
" dtype=int32)>}"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"vectorizer(['I love transformers'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"使用与原始网络训练时相同的向量化工具是非常重要的。此外，BERT 向量化工具会返回三个组件：\n",
|
||||
"* `input_word_ids`，这是输入句子的标记编号序列\n",
|
||||
"* `input_mask`，用于显示序列中哪些部分是实际输入，哪些部分是填充。这与 `Masking` 层生成的掩码类似\n",
|
||||
"* `input_type_ids`，用于语言建模任务，允许在一个序列中指定两个输入句子。\n",
|
||||
"\n",
|
||||
"然后，我们可以实例化 BERT 特征提取器：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"bert = hub.KerasLayer('https://tfhub.dev/tensorflow/small_bert/bert_en_uncased_L-4_H-128_A-2/1')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"pooled_output -> (1, 128)\n",
|
||||
"encoder_outputs -> 4\n",
|
||||
"sequence_output -> (1, 128, 128)\n",
|
||||
"default -> (1, 128)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"z = bert(vectorizer(['I love transformers']))\n",
|
||||
"for i,x in z.items():\n",
|
||||
" print(f\"{i} -> { len(x) if isinstance(x, list) else x.shape }\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"因此，BERT 层会返回一些有用的结果：\n",
|
||||
"* `pooled_output` 是通过对序列中所有标记进行平均得到的结果。你可以将其视为整个网络的智能语义嵌入。它等同于我们之前模型中 `GlobalAveragePooling1D` 层的输出。\n",
|
||||
"* `sequence_output` 是最后一个 Transformer 层的输出（对应于我们上面模型中的 `TransformerBlock` 的输出）。\n",
|
||||
"* `encoder_outputs` 是所有 Transformer 层的输出。由于我们加载了一个 4 层的 BERT 模型（你可能已经从名字中包含的 `4_H` 猜到了），它有 4 个张量。最后一个张量与 `sequence_output` 相同。\n",
|
||||
"\n",
|
||||
"现在我们将定义端到端的分类模型。我们将使用*函数式模型定义*，在定义模型输入后，通过一系列表达式计算其输出。我们还会将 BERT 模型的权重设置为不可训练，仅训练最终的分类器：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"model\"\n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"Layer (type) Output Shape Param # Connected to \n",
|
||||
"==================================================================================================\n",
|
||||
"input_1 (InputLayer) [(None,)] 0 \n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"keras_layer (KerasLayer) {'input_type_ids': ( 0 input_1[0][0] \n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"keras_layer_1 (KerasLayer) {'pooled_output': (N 4782465 keras_layer[0][0] \n",
|
||||
" keras_layer[0][1] \n",
|
||||
" keras_layer[0][2] \n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"dropout_4 (Dropout) (None, 128) 0 keras_layer_1[0][5] \n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"dense_4 (Dense) (None, 4) 516 dropout_4[0][0] \n",
|
||||
"==================================================================================================\n",
|
||||
"Total params: 4,782,981\n",
|
||||
"Trainable params: 516\n",
|
||||
"Non-trainable params: 4,782,465\n",
|
||||
"__________________________________________________________________________________________________\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"inp = keras.Input(shape=(),dtype=tf.string)\n",
|
||||
"x = vectorizer(inp)\n",
|
||||
"x = bert(x)\n",
|
||||
"x = keras.layers.Dropout(0.1)(x['pooled_output'])\n",
|
||||
"out = keras.layers.Dense(4,activation='softmax')(x)\n",
|
||||
"model = keras.models.Model(inp,out)\n",
|
||||
"bert.trainable = False\n",
|
||||
"model.summary()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"938/938 [==============================] - 528s 559ms/step - loss: 0.8056 - acc: 0.6983 - val_loss: 0.5953 - val_acc: 0.7888\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7f9bb1e36d00>"
|
||||
]
|
||||
},
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(128),validation_data=ds_test.map(tupelize).batch(128))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"尽管可训练参数很少，但过程仍然相当缓慢，因为 BERT 特征提取器计算量很大。看起来我们无法达到合理的准确性，可能是由于训练不足或模型参数不足。\n",
|
||||
"\n",
|
||||
"让我们尝试解冻 BERT 的权重并对其进行训练。这需要非常小的学习率，同时还需要更谨慎的训练策略，包括**预热**，并使用 **AdamW** 优化器。我们将使用 `tf-models-official` 包来创建优化器：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"model\"\n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"Layer (type) Output Shape Param # Connected to \n",
|
||||
"==================================================================================================\n",
|
||||
"input_1 (InputLayer) [(None,)] 0 \n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"keras_layer (KerasLayer) {'input_type_ids': ( 0 input_1[0][0] \n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"keras_layer_1 (KerasLayer) {'pooled_output': (N 4782465 keras_layer[0][0] \n",
|
||||
" keras_layer[0][1] \n",
|
||||
" keras_layer[0][2] \n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"dropout_4 (Dropout) (None, 128) 0 keras_layer_1[0][5] \n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"dense_4 (Dense) (None, 4) 516 dropout_4[0][0] \n",
|
||||
"==================================================================================================\n",
|
||||
"Total params: 4,782,981\n",
|
||||
"Trainable params: 4,782,980\n",
|
||||
"Non-trainable params: 1\n",
|
||||
"__________________________________________________________________________________________________\n",
|
||||
"938/938 [==============================] - 629s 664ms/step - loss: 0.6344 - acc: 0.7658 - val_loss: 0.4876 - val_acc: 0.8247\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7f9bb0bd0070>"
|
||||
]
|
||||
},
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from official.nlp import optimization \n",
|
||||
"bert.trainable=True\n",
|
||||
"model.summary()\n",
|
||||
"epochs = 3\n",
|
||||
"opt = optimization.create_optimizer(\n",
|
||||
" init_lr=3e-5,\n",
|
||||
" num_train_steps=epochs*len(ds_train),\n",
|
||||
" num_warmup_steps=0.1*epochs*len(ds_train),\n",
|
||||
" optimizer_type='adamw')\n",
|
||||
"\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer=opt)\n",
|
||||
"model.fit(ds_train.map(tupelize).batch(128),validation_data=ds_test.map(tupelize).batch(128))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"正如你所看到的，训练过程相当缓慢——但你可能想尝试训练模型几个周期（5-10），看看与我们之前使用的方法相比，是否能获得最佳结果。\n",
|
||||
"\n",
|
||||
"## Huggingface Transformers库\n",
|
||||
"\n",
|
||||
"另一种非常常见（且稍微简单一些）的使用Transformer模型的方法是[HuggingFace包](https://github.com/huggingface/)，它为不同的NLP任务提供了简单的构建模块。该库同时支持Tensorflow和PyTorch，这两个非常流行的神经网络框架。\n",
|
||||
"\n",
|
||||
"> **注意**：如果你对了解Transformers库的工作原理不感兴趣——可以直接跳到笔记本的最后部分，因为你不会看到与我们之前所做的有实质性不同的内容。我们将重复使用不同的库和更大的模型来训练BERT模型的相同步骤。因此，这个过程涉及一些相当长时间的训练，所以你可能只需要浏览代码即可。\n",
|
||||
"\n",
|
||||
"让我们看看如何使用[Huggingface Transformers](http://huggingface.co)来解决我们的问题。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"首先，我们需要选择将要使用的模型。除了内置的一些模型之外，Huggingface还提供了一个[在线模型库](https://huggingface.co/models)，社区中有许多预训练模型可供选择。所有这些模型都可以通过提供模型名称来加载和使用。模型所需的所有二进制文件会自动下载。\n",
|
||||
"\n",
|
||||
"有时候，你可能需要加载自己的模型，这种情况下可以指定包含所有相关文件的目录，包括分词器的参数、包含模型参数的`config.json`文件、二进制权重等。\n",
|
||||
"\n",
|
||||
"通过模型名称，我们可以实例化模型和分词器。让我们从分词器开始：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import transformers\n",
|
||||
"\n",
|
||||
"# To load the model from Internet repository using model name. \n",
|
||||
"# Use this if you are running from your own copy of the notebooks\n",
|
||||
"bert_model = 'bert-base-uncased' \n",
|
||||
"\n",
|
||||
"# To load the model from the directory on disk. Use this for Microsoft Learn module, because we have\n",
|
||||
"# prepared all required files for you.\n",
|
||||
"#bert_model = './bert'\n",
|
||||
"\n",
|
||||
"tokenizer = transformers.BertTokenizer.from_pretrained(bert_model)\n",
|
||||
"\n",
|
||||
"MAX_SEQ_LEN = 128\n",
|
||||
"PAD_INDEX = tokenizer.convert_tokens_to_ids(tokenizer.pad_token)\n",
|
||||
"UNK_INDEX = tokenizer.convert_tokens_to_ids(tokenizer.unk_token)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"`tokenizer` 对象包含 `encode` 函数，可直接用于编码文本：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[101, 23435, 12314, 2003, 1037, 2307, 7705, 2005, 17953, 2361, 102]"
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"tokenizer.encode('Tensorflow is a great framework for NLP')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们还可以使用分词器以适合传递给模型的方式对序列进行编码，例如包括 `token_ids`、`input_mask` 字段等。我们还可以通过提供 `return_tensors='tf'` 参数来指定我们需要 Tensorflow 张量：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'input_ids': <tf.Tensor: shape=(1, 5), dtype=int32, numpy=array([[ 101, 7592, 1010, 2045, 102]], dtype=int32)>, 'token_type_ids': <tf.Tensor: shape=(1, 5), dtype=int32, numpy=array([[0, 0, 0, 0, 0]], dtype=int32)>, 'attention_mask': <tf.Tensor: shape=(1, 5), dtype=int32, numpy=array([[1, 1, 1, 1, 1]], dtype=int32)>}"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"tokenizer(['Hello, there'],return_tensors='tf')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在我们的案例中，我们将使用预训练的BERT模型，名为`bert-base-uncased`。*Uncased*表示该模型对大小写不敏感。\n",
|
||||
"\n",
|
||||
"在训练模型时，我们需要提供经过分词的序列作为输入，因此我们将设计数据处理管道。由于`tokenizer.encode`是一个Python函数，我们将采用与上一单元相同的方法，通过调用`py_function`来实现：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 31,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def process(x):\n",
|
||||
" return tokenizer.encode(x.numpy().decode('utf-8'),return_tensors='tf',padding='max_length',max_length=MAX_SEQ_LEN,truncation=True)[0]\n",
|
||||
"\n",
|
||||
"def process_fn(x):\n",
|
||||
" s = x['title']+' '+x['description']\n",
|
||||
" e = tf.py_function(process,inp=[s],Tout=(tf.int32))\n",
|
||||
" e.set_shape(MAX_SEQ_LEN)\n",
|
||||
" return e,x['label']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在我们可以使用`BertForSequenceClassification`包加载实际模型。这确保了我们的模型已经具有分类所需的架构，包括最终的分类器。您会看到一条警告消息，指出最终分类器的权重尚未初始化，模型需要进行预训练——这完全没问题，因为这正是我们即将要做的！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 32,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"model = transformers.TFBertForSequenceClassification.from_pretrained(bert_model,num_labels=4,output_attentions=False)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 33,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"tf_bert_for_sequence_classification_1\"\n",
|
||||
"_________________________________________________________________\n",
|
||||
"Layer (type) Output Shape Param # \n",
|
||||
"=================================================================\n",
|
||||
"bert (TFBertMainLayer) multiple 109482240 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"dropout_75 (Dropout) multiple 0 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"classifier (Dense) multiple 3076 \n",
|
||||
"=================================================================\n",
|
||||
"Total params: 109,485,316\n",
|
||||
"Trainable params: 109,485,316\n",
|
||||
"Non-trainable params: 0\n",
|
||||
"_________________________________________________________________\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.summary()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"从 `summary()` 可以看出，该模型包含将近 1.1 亿个参数！据推测，如果我们希望在相对较小的数据集上进行简单的分类任务，我们不希望训练 BERT 基础层：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 34,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"tf_bert_for_sequence_classification_1\"\n",
|
||||
"_________________________________________________________________\n",
|
||||
"Layer (type) Output Shape Param # \n",
|
||||
"=================================================================\n",
|
||||
"bert (TFBertMainLayer) multiple 109482240 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"dropout_75 (Dropout) multiple 0 \n",
|
||||
"_________________________________________________________________\n",
|
||||
"classifier (Dense) multiple 3076 \n",
|
||||
"=================================================================\n",
|
||||
"Total params: 109,485,316\n",
|
||||
"Trainable params: 3,076\n",
|
||||
"Non-trainable params: 109,482,240\n",
|
||||
"_________________________________________________________________\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.layers[0].trainable = False\n",
|
||||
"model.summary()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在我们可以开始训练了！\n",
|
||||
"\n",
|
||||
"> **注意**：训练完整规模的BERT模型可能会非常耗时！因此，我们只会训练前32个批次。这只是为了展示模型训练的设置方式。如果你有兴趣尝试完整规模的训练，只需移除 `steps_per_epoch` 和 `validation_steps` 参数，然后准备好等待吧！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 30,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"32/32 [==============================] - 142s 4s/step - loss: 1.3896 - acc: 0.2500 - val_loss: 1.3863 - val_acc: 0.2480\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<tensorflow.python.keras.callbacks.History at 0x7f1d40a4b6a0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 30,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.compile('adam','sparse_categorical_crossentropy',['acc'])\n",
|
||||
"tf.get_logger().setLevel('ERROR')\n",
|
||||
"model.fit(ds_train.map(process_fn).batch(32),validation_data=ds_test.map(process_fn).batch(32),steps_per_epoch=32,validation_steps=2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"如果你增加迭代次数并等待足够长的时间，同时进行多个训练周期，你可以预期 BERT 分类会给我们带来最佳的准确率！这是因为 BERT 已经对语言结构有了相当好的理解，我们只需要微调最终的分类器即可。然而，由于 BERT 是一个大型模型，整个训练过程需要很长时间，并且需要强大的计算能力！（GPU，最好是多块 GPU）。\n",
|
||||
"\n",
|
||||
"> **Note:** 在我们的示例中，我们使用的是最小的预训练 BERT 模型之一。还有更大的模型可能会带来更好的结果。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 要点\n",
|
||||
"\n",
|
||||
"在本单元中，我们了解了基于**transformers**的最新模型架构。我们将其应用于文本分类任务，但同样，BERT模型也可以用于实体提取、问答以及其他自然语言处理任务。\n",
|
||||
"\n",
|
||||
"Transformer模型代表了当前自然语言处理领域的最先进技术。在大多数情况下，当你开始尝试实现定制的自然语言处理解决方案时，它应该是首选。然而，如果你希望构建高级的神经网络模型，理解本模块中讨论的循环神经网络的基本原理是非常重要的。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。原始语言的文档应被视为权威来源。对于关键信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "0cb620c6d4b9f7a635928804c26cf22403d89d98d79684e4529119355ee6d5a5"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "py38_tensorflow",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "ab59c532409774988ab875f2260e8e53",
|
||||
"translation_date": "2025-08-31T10:42:25+00:00",
|
||||
"source_file": "lessons/5-NLP/18-Transformers/TransformersTF.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# 作业：Transformers
|
||||
|
||||
在 HuggingFace 上尝试使用 Transformers！试试他们提供的一些脚本，来操作他们网站上提供的各种模型：https://huggingface.co/docs/transformers/run_scripts。尝试使用他们的一个数据集，然后从本课程或 Kaggle 导入一个你自己的数据集，看看你是否能生成有趣的文本。制作一个包含你发现的内容的笔记本。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,492 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 命名实体识别 (NER)\n",
|
||||
"\n",
|
||||
"本笔记本来自 [AI for Beginners Curriculum](http://aka.ms/ai-beginners)。\n",
|
||||
"\n",
|
||||
"在这个示例中，我们将学习如何在 [命名实体识别的标注语料库](https://www.kaggle.com/datasets/abhinavwalia95/entity-annotated-corpus) 数据集上训练 NER 模型。在开始之前，请将 [ner_dataset.csv](https://www.kaggle.com/datasets/abhinavwalia95/entity-annotated-corpus?resource=download&select=ner_dataset.csv) 文件下载到当前目录中。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 62,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"from tensorflow import keras\n",
|
||||
"import numpy as np"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 准备数据集\n",
|
||||
"\n",
|
||||
"我们将从将数据集读取到一个数据框开始。如果你想了解更多关于使用 Pandas 的内容，可以访问我们的[数据科学入门](http://aka.ms/datascience-beginners)中的[数据处理课程](https://github.com/microsoft/Data-Science-For-Beginners/tree/main/2-Working-With-Data/07-python)。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>Sentence #</th>\n",
|
||||
" <th>Word</th>\n",
|
||||
" <th>POS</th>\n",
|
||||
" <th>Tag</th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>0</th>\n",
|
||||
" <td>Sentence: 1</td>\n",
|
||||
" <td>Thousands</td>\n",
|
||||
" <td>NNS</td>\n",
|
||||
" <td>O</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>1</th>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>of</td>\n",
|
||||
" <td>IN</td>\n",
|
||||
" <td>O</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2</th>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>demonstrators</td>\n",
|
||||
" <td>NNS</td>\n",
|
||||
" <td>O</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>3</th>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>have</td>\n",
|
||||
" <td>VBP</td>\n",
|
||||
" <td>O</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>4</th>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>marched</td>\n",
|
||||
" <td>VBN</td>\n",
|
||||
" <td>O</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" Sentence # Word POS Tag\n",
|
||||
"0 Sentence: 1 Thousands NNS O\n",
|
||||
"1 NaN of IN O\n",
|
||||
"2 NaN demonstrators NNS O\n",
|
||||
"3 NaN have VBP O\n",
|
||||
"4 NaN marched VBN O"
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"df = pd.read_csv('ner_dataset.csv',encoding='unicode-escape')\n",
|
||||
"df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"让我们获取唯一标签并创建查找字典，以便我们可以将标签转换为类别编号：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array(['O', 'B-geo', 'B-gpe', 'B-per', 'I-geo', 'B-org', 'I-org', 'B-tim',\n",
|
||||
" 'B-art', 'I-art', 'I-per', 'I-gpe', 'I-tim', 'B-nat', 'B-eve',\n",
|
||||
" 'I-eve', 'I-nat'], dtype=object)"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"tags = df.Tag.unique()\n",
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"'O'"
|
||||
]
|
||||
},
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"id2tag = dict(enumerate(tags))\n",
|
||||
"tag2id = { v : k for k,v in id2tag.items() }\n",
|
||||
"\n",
|
||||
"id2tag[0]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在我们需要对词汇做同样的处理。为简单起见，我们将在不考虑词频的情况下创建词汇表；在实际应用中，您可能需要使用 Keras 向量化工具，并限制单词的数量。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"vocab = set(df['Word'].apply(lambda x: x.lower()))\n",
|
||||
"id2word = { i+1 : v for i,v in enumerate(vocab) }\n",
|
||||
"id2word[0] = '<UNK>'\n",
|
||||
"vocab.add('<UNK>')\n",
|
||||
"word2id = { v : k for k,v in id2word.items() }"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们需要创建一个句子数据集用于训练。让我们遍历原始数据集，并将所有单独的句子分离为 `X`（单词列表）和 `Y`（标记列表）：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 41,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"X,Y = [],[]\n",
|
||||
"s,t = [],[]\n",
|
||||
"for i,row in df[['Sentence #','Word','Tag']].iterrows():\n",
|
||||
" if pd.isna(row['Sentence #']):\n",
|
||||
" s.append(row['Word'])\n",
|
||||
" t.append(row['Tag'])\n",
|
||||
" else:\n",
|
||||
" if len(s)>0:\n",
|
||||
" X.append(s)\n",
|
||||
" Y.append(t)\n",
|
||||
" s,t = [row['Word']],[row['Tag']]\n",
|
||||
"X.append(s)\n",
|
||||
"Y.append(t)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 93,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"([10386,\n",
|
||||
" 23515,\n",
|
||||
" 4134,\n",
|
||||
" 29620,\n",
|
||||
" 7954,\n",
|
||||
" 13583,\n",
|
||||
" 21193,\n",
|
||||
" 12222,\n",
|
||||
" 27322,\n",
|
||||
" 18258,\n",
|
||||
" 5815,\n",
|
||||
" 15880,\n",
|
||||
" 5355,\n",
|
||||
" 25242,\n",
|
||||
" 31327,\n",
|
||||
" 18258,\n",
|
||||
" 27067,\n",
|
||||
" 23515,\n",
|
||||
" 26444,\n",
|
||||
" 14412,\n",
|
||||
" 358,\n",
|
||||
" 26551,\n",
|
||||
" 5011,\n",
|
||||
" 30558],\n",
|
||||
" [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0])"
|
||||
]
|
||||
},
|
||||
"execution_count": 93,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def vectorize(seq):\n",
|
||||
" return [word2id[x.lower()] for x in seq]\n",
|
||||
"\n",
|
||||
"def tagify(seq):\n",
|
||||
" return [tag2id[x] for x in seq]\n",
|
||||
"\n",
|
||||
"Xv = list(map(vectorize,X))\n",
|
||||
"Yv = list(map(tagify,Y))\n",
|
||||
"\n",
|
||||
"Xv[0], Yv[0]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"为了简单起见，我们将所有句子用0标记填充到最大长度。在现实生活中，我们可能希望使用更聪明的策略，仅在一个小批量内填充序列。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 51,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"X_data = keras.preprocessing.sequence.pad_sequences(Xv,padding='post')\n",
|
||||
"Y_data = keras.preprocessing.sequence.pad_sequences(Yv,padding='post')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 定义标记分类网络\n",
|
||||
"\n",
|
||||
"我们将使用两层双向 LSTM 网络进行标记分类。为了对最后一层 LSTM 的每个输出应用密集分类器，我们将使用 `TimeDistributed` 构造，它会在每一步将相同的密集层复制到 LSTM 的每个输出：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 94,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Model: \"sequential_3\"\n",
|
||||
"_________________________________________________________________\n",
|
||||
" Layer (type) Output Shape Param # \n",
|
||||
"=================================================================\n",
|
||||
" embedding_4 (Embedding) (None, 104, 300) 9545400 \n",
|
||||
" \n",
|
||||
" bidirectional_6 (Bidirectio (None, 104, 200) 320800 \n",
|
||||
" nal) \n",
|
||||
" \n",
|
||||
" bidirectional_7 (Bidirectio (None, 104, 200) 240800 \n",
|
||||
" nal) \n",
|
||||
" \n",
|
||||
" time_distributed_3 (TimeDis (None, 104, 17) 3417 \n",
|
||||
" tributed) \n",
|
||||
" \n",
|
||||
"=================================================================\n",
|
||||
"Total params: 10,110,417\n",
|
||||
"Trainable params: 10,110,417\n",
|
||||
"Non-trainable params: 0\n",
|
||||
"_________________________________________________________________\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"maxlen = X_data.shape[1]\n",
|
||||
"vocab_size = len(vocab)\n",
|
||||
"num_tags = len(tags)\n",
|
||||
"model = keras.models.Sequential([\n",
|
||||
" keras.layers.Embedding(vocab_size, 300, input_length=maxlen),\n",
|
||||
" keras.layers.Bidirectional(keras.layers.LSTM(units=100, activation='tanh', return_sequences=True)),\n",
|
||||
" keras.layers.Bidirectional(keras.layers.LSTM(units=100, activation='tanh', return_sequences=True)),\n",
|
||||
" keras.layers.TimeDistributed(keras.layers.Dense(num_tags, activation='softmax'))\n",
|
||||
"])\n",
|
||||
"model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['acc'])\n",
|
||||
"model.summary()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"请注意，这里我们为数据集明确指定了 `maxlen`——如果我们希望网络能够处理可变长度的序列，那么在定义网络时需要更巧妙一些。\n",
|
||||
"\n",
|
||||
"现在让我们开始训练模型。为了加快速度，我们只训练一个周期，但你可以尝试训练更长时间。此外，你可能希望将数据集的一部分分离出来作为训练数据集，以观察验证准确率。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 57,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"1499/1499 [==============================] - 740s 488ms/step - loss: 0.0667 - acc: 0.9841\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<keras.callbacks.History at 0x16f0bb2a310>"
|
||||
]
|
||||
},
|
||||
"execution_count": 57,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model.fit(X_data,Y_data)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 测试结果\n",
|
||||
"\n",
|
||||
"现在让我们看看实体识别模型在一个示例句子上的表现：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 91,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sent = 'John Smith went to Paris to attend a conference in cancer development institute'\n",
|
||||
"words = sent.lower().split()\n",
|
||||
"v = keras.preprocessing.sequence.pad_sequences([[word2id[x] for x in words]],padding='post',maxlen=maxlen)\n",
|
||||
"res = model(v)[0]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 92,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"john -> B-per\n",
|
||||
"smith -> I-per\n",
|
||||
"went -> O\n",
|
||||
"to -> O\n",
|
||||
"paris -> B-geo\n",
|
||||
"to -> O\n",
|
||||
"attend -> O\n",
|
||||
"a -> O\n",
|
||||
"conference -> O\n",
|
||||
"in -> O\n",
|
||||
"cancer -> B-org\n",
|
||||
"development -> I-org\n",
|
||||
"institute -> I-org\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"r = np.argmax(res.numpy(),axis=1)\n",
|
||||
"for i,w in zip(r,words):\n",
|
||||
" print(f\"{w} -> {id2tag[i]}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 关键点\n",
|
||||
"\n",
|
||||
"即使是简单的 LSTM 模型在命名实体识别（NER）任务中也能表现出不错的效果。然而，如果想要获得更好的结果，可以考虑使用大型预训练语言模型，例如 BERT。使用 Huggingface Transformers 库训练 BERT 进行 NER 的方法可以在[这里](https://huggingface.co/course/chapter7/2?fw=pt)找到。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.12 ('py38')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"coopTranslator": {
|
||||
"original_hash": "254d25052dcca4ef84f59a05f2935bdc",
|
||||
"translation_date": "2025-08-31T10:45:16+00:00",
|
||||
"source_file": "lessons/5-NLP/19-NER/NER-TF.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,85 @@
|
|||
# 命名实体识别
|
||||
|
||||
到目前为止，我们主要集中在一个自然语言处理（NLP）任务——分类。然而，神经网络还可以完成其他的NLP任务，其中之一就是**[命名实体识别](https://wikipedia.org/wiki/Named-entity_recognition)**（NER），它处理的是识别文本中的特定实体，例如地点、人物姓名、日期时间区间、化学式等等。
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/37)
|
||||
|
||||
## 使用NER的示例
|
||||
|
||||
假设你想开发一个类似于亚马逊Alexa或谷歌助手的自然语言聊天机器人。智能聊天机器人的工作方式是通过对输入句子进行文本分类来“理解”用户的需求。分类的结果是所谓的**意图**，它决定了聊天机器人应该执行什么操作。
|
||||
|
||||
<img alt="Bot NER" src="../../../../../translated_images/zh-CN/bot-ner.4b09235dbb0ad275.webp" width="50%"/>
|
||||
|
||||
> 图片由作者提供
|
||||
|
||||
然而，用户可能会在短语中提供一些参数。例如，当询问天气时，她可能会指定一个地点或日期。机器人应该能够理解这些实体，并在执行操作之前相应地填充参数槽。这正是NER的作用所在。
|
||||
|
||||
> ✅ 另一个例子是[分析科学医学论文](https://soshnikov.com/science/analyzing-medical-papers-with-azure-and-text-analytics-for-health/)。我们需要寻找的主要内容是特定的医学术语，例如疾病和药物成分。虽然少量疾病可能可以通过子字符串搜索提取，但更复杂的实体，例如化学化合物和药物名称，则需要更复杂的方法。
|
||||
|
||||
## NER作为标记分类
|
||||
|
||||
NER模型本质上是**标记分类模型**，因为对于每个输入标记，我们需要决定它是否属于某个实体，如果属于，则属于哪个实体类别。
|
||||
|
||||
考虑以下论文标题：
|
||||
|
||||
**三尖瓣反流**和**碳酸锂**在新生儿中的**毒性**。
|
||||
|
||||
这里的实体是：
|
||||
|
||||
* 三尖瓣反流是一种疾病（`DIS`）
|
||||
* 碳酸锂是一种化学物质（`CHEM`）
|
||||
* 毒性也是一种疾病（`DIS`）
|
||||
|
||||
注意，一个实体可能跨越多个标记。而且，如本例所示，我们需要区分两个连续的实体。因此，通常为每个实体使用两个类别——一个指定实体的第一个标记（通常使用`B-`前缀，表示**开始**），另一个表示实体的延续部分（`I-`，表示**内部标记**）。我们还使用`O`作为类别来表示所有**其他**标记。这种标记标注称为[BIO标注](https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging))（或IOB）。标注后的标题如下所示：
|
||||
|
||||
标记 | 标签
|
||||
------|-----
|
||||
三尖瓣 | B-DIS
|
||||
反流 | I-DIS
|
||||
和 | O
|
||||
碳酸 | B-CHEM
|
||||
锂 | I-CHEM
|
||||
毒性 | B-DIS
|
||||
在 | O
|
||||
新生儿 | O
|
||||
中 | O
|
||||
的 | O
|
||||
毒性 | O
|
||||
。 | O
|
||||
|
||||
由于我们需要在标记和类别之间建立一一对应关系，我们可以从下图中训练一个最右侧的**多对多**神经网络模型：
|
||||
|
||||
|
||||
|
||||
> *图片来自[这篇博客文章](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)作者为[Andrej Karpathy](http://karpathy.github.io/)。NER标记分类模型对应于此图片中最右侧的网络架构。*
|
||||
|
||||
## 训练NER模型
|
||||
|
||||
由于NER模型本质上是一个标记分类模型，我们可以使用我们已经熟悉的RNN来完成这个任务。在这种情况下，每个循环网络块将返回标记ID。以下示例笔记本展示了如何训练用于标记分类的LSTM。
|
||||
|
||||
## ✍️ 示例笔记本：NER
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
* [使用TensorFlow进行NER](NER-TF.ipynb)
|
||||
|
||||
## 总结
|
||||
|
||||
NER模型是一种**标记分类模型**，这意味着它可以用于执行标记分类。这是NLP中非常常见的任务，帮助识别文本中的特定实体，包括地点、姓名、日期等。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
完成以下链接的作业，训练一个用于医学术语的命名实体识别模型，然后尝试在不同的数据集上使用它。
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/38)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
阅读博客[循环神经网络的非凡有效性](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)，并按照文章中的进一步阅读部分加深你的知识。
|
||||
|
||||
## [作业](lab/README.md)
|
||||
|
||||
在本课的作业中，你需要训练一个医学实体识别模型。你可以从本课中描述的LSTM模型训练开始，然后使用BERT变换器模型。阅读[说明](lab/README.md)以获取所有详细信息。
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
# 命名实体识别 (NER)
|
||||
|
||||
来自 [AI for Beginners Curriculum](https://github.com/microsoft/ai-for-beginners) 的实验任务。
|
||||
|
||||
## 任务
|
||||
|
||||
在本实验中，你需要训练一个用于医学术语的命名实体识别模型。
|
||||
|
||||
## 数据集
|
||||
|
||||
为了训练 NER 模型，我们需要一个标注良好的包含医学实体的数据集。[BC5CDR 数据集](https://biocreative.bioinformatics.udel.edu/tasks/biocreative-v/track-3-cdr/) 包含了从 1500 多篇论文中提取的疾病和化学物质实体。你可以在其网站注册后下载该数据集。
|
||||
|
||||
BC5CDR 数据集的格式如下：
|
||||
|
||||
```
|
||||
6794356|t|Tricuspid valve regurgitation and lithium carbonate toxicity in a newborn infant.
|
||||
6794356|a|A newborn with massive tricuspid regurgitation, atrial flutter, congestive heart failure, and a high serum lithium level is described. This is the first patient to initially manifest tricuspid regurgitation and atrial flutter, and the 11th described patient with cardiac disease among infants exposed to lithium compounds in the first trimester of pregnancy. Sixty-three percent of these infants had tricuspid valve involvement. Lithium carbonate may be a factor in the increasing incidence of congenital heart disease when taken during early pregnancy. It also causes neurologic depression, cyanosis, and cardiac arrhythmia when consumed prior to delivery.
|
||||
6794356 0 29 Tricuspid valve regurgitation Disease D014262
|
||||
6794356 34 51 lithium carbonate Chemical D016651
|
||||
6794356 52 60 toxicity Disease D064420
|
||||
...
|
||||
```
|
||||
|
||||
在这个数据集中，第一行和第二行分别是论文标题和摘要，接下来是各个实体的信息，包括它们在标题+摘要块中的起始和结束位置。除了实体类型外，你还可以获得该实体在某些医学本体中的本体 ID。
|
||||
|
||||
你需要编写一些 Python 代码，将这些数据转换为 BIO 编码格式。
|
||||
|
||||
## 网络
|
||||
|
||||
第一次尝试 NER 时，可以使用 LSTM 网络，就像你在课程中看到的示例一样。然而，在 NLP 任务中，[Transformer 架构](https://en.wikipedia.org/wiki/Transformer_(machine_learning_model))，特别是 [BERT 语言模型](https://en.wikipedia.org/wiki/BERT_(language_model))，通常能取得更好的效果。预训练的 BERT 模型能够理解语言的基本结构，并且可以通过相对较小的数据集和计算成本针对特定任务进行微调。
|
||||
|
||||
由于我们计划将 NER 应用于医学场景，因此使用在医学文本上训练的 BERT 模型是合理的。微软研究院发布了一个名为 [PubMedBERT][PubMedBERT] 的预训练模型（[相关论文][PubMedBERT-Pub]），该模型使用 [PubMed](https://pubmed.ncbi.nlm.nih.gov/) 仓库中的文本进行了微调。
|
||||
|
||||
训练 Transformer 模型的事实标准是 [Hugging Face Transformers](https://huggingface.co/) 库。该库还包含一个由社区维护的预训练模型仓库，其中包括 PubMedBERT。加载和使用该模型只需要几行代码：
|
||||
|
||||
```python
|
||||
model_name = "microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract"
|
||||
classes = ... # number of classes: 2*entities+1
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||||
model = BertForTokenClassification.from_pretrained(model_name, classes)
|
||||
```
|
||||
|
||||
这段代码为我们提供了一个 `model` 对象，用于基于 `classes` 数量的类别进行标注任务，以及一个 `tokenizer` 对象，用于将输入文本分割为 token。你需要将数据集转换为 BIO 格式，同时考虑到 PubMedBERT 的分词方式。你可以参考 [这段 Python 代码](https://gist.github.com/shwars/580b55684be3328eb39ecf01b9cbbd88) 来获取灵感。
|
||||
|
||||
## 收获
|
||||
|
||||
这个任务非常接近实际工作中可能遇到的任务，尤其是当你希望从大量自然语言文本中获取更多洞见时。在我们的案例中，我们可以将训练好的模型应用于 [COVID 相关论文数据集](https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge)，看看能从中获取哪些洞见。[这篇博客文章](https://soshnikov.com/science/analyzing-medical-papers-with-azure-and-text-analytics-for-health/) 和 [这篇论文](https://www.mdpi.com/2504-2289/6/1/4) 描述了可以使用 NER 对该论文语料库进行的研究。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。因使用本翻译而引起的任何误解或误读，我们概不负责。
|
||||
|
|
@ -0,0 +1,325 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 探索 OpenAI GPT\n",
|
||||
"\n",
|
||||
"本笔记本是 [AI 初学者课程](http://aka.ms/ai-beginners) 的一部分。\n",
|
||||
"\n",
|
||||
"在本笔记本中，我们将使用 Hugging Face 的 `transformers` 库来探索如何与 OpenAI-GPT 模型进行交互。\n",
|
||||
"\n",
|
||||
"话不多说，让我们创建文本生成管道并开始生成吧！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"c:\\Users\\bethanycheum\\Desktop\\AI-For-Beginners\\.venv\\lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
|
||||
" from .autonotebook import tqdm as notebook_tqdm\n",
|
||||
"Downloading model.safetensors: 100%|██████████| 479M/479M [04:28<00:00, 1.78MB/s] \n",
|
||||
"c:\\Users\\bethanycheum\\Desktop\\AI-For-Beginners\\.venv\\lib\\site-packages\\huggingface_hub\\file_download.py:133: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\\Users\\bethanycheum\\.cache\\huggingface\\hub. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.\n",
|
||||
"To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development\n",
|
||||
" warnings.warn(message)\n",
|
||||
"Some weights of OpenAIGPTLMHeadModel were not initialized from the model checkpoint at openai-gpt and are newly initialized: ['position_ids']\n",
|
||||
"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n",
|
||||
"Downloading (…)neration_config.json: 100%|██████████| 74.0/74.0 [00:00<00:00, 48.8kB/s]\n",
|
||||
"Downloading (…)olve/main/vocab.json: 100%|██████████| 816k/816k [00:00<00:00, 1.76MB/s]\n",
|
||||
"Downloading (…)olve/main/merges.txt: 100%|██████████| 458k/458k [00:00<00:00, 1.11MB/s]\n",
|
||||
"Downloading (…)/main/tokenizer.json: 100%|██████████| 1.27M/1.27M [00:00<00:00, 2.12MB/s]\n",
|
||||
"Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers\n",
|
||||
"pip install xformers.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'generated_text': \"Hello! I am a neural network, and I want to say that i apologize for not coming to you yourself, for not helping you, and that i was too busy getting dressed and studying for a midterm. you know, the kind where the teachers are like that and they come in pairs with their boyfriends, but not with theirs. it's true, that i have had a girlfriend, and i'm only going on wednesdays and thursdays because i was too busy with college, but maybe\"},\n",
|
||||
" {'generated_text': 'Hello! I am a neural network, and I want to say that we have been blessed with a wonderful gift ; no one of us has died at all. and our spirits are strong, very strong. in one very lucky moment of luck for you, all has been given direction and destiny, and for us there are no more mysteries. the earth has been chosen for you, and that earth is now ours, and you must be forever in our hearts. \" \\n the words, as one,'},\n",
|
||||
" {'generated_text': 'Hello! I am a neural network, and I want to say that if you would just turn and face the general, you would have a nice day. \" \\n \" sure thing, \" said one of the soldiers, and started to run. the rest of the soldiers followed, shouting. the general turned to general zulu, raising his arm. the general said something in his native language, and the general immediately started to run. zulu started to move toward the wall, with the'},\n",
|
||||
" {'generated_text': 'Hello! I am a neural network, and I want to say that i am not a doctor but an anthropologist to you, a specialist, a specialist in the field of astrobiological biology, and that i am very much involved in this investigation. i am not sure, i am not certain, but i can confirm your conclusions and therefore i will go to the top. i have a colleague who has just returned from this expedition and his findings confirm that you are a specialist. that is, he'},\n",
|
||||
" {'generated_text': \"Hello! I am a neural network, and I want to say that everyone here is in agreement that no matter how many times i say to myself,'he was never a man of action on the battlefield,'or'he 'll never take a chance at killing any civilians,'or'he 'll never let his men go undefended against enemy forces of this caliber,'or'that's just what i need in a day like today. \\n you see, there are only three groups that\"}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from transformers import pipeline\n",
|
||||
"\n",
|
||||
"model_name = 'openai-gpt' \n",
|
||||
"\n",
|
||||
"generator = pipeline('text-generation', model=model_name)\n",
|
||||
"\n",
|
||||
"generator(\"Hello! I am a neural network, and I want to say that\", max_length=100, num_return_sequences=5)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 提示工程\n",
|
||||
"\n",
|
||||
"在某些问题中，你可以通过设计正确的提示直接使用 openai-gpt 生成内容。请看以下示例：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'generated_text': 'Synonyms of a word cat: the same cat i used to stare at, and you in'},\n",
|
||||
" {'generated_text': 'Synonyms of a word cat: cat of the woods, cat of the hills, cat of'},\n",
|
||||
" {'generated_text': 'Synonyms of a word cat: you! \\n \" it\\'s a girl. \" i said'},\n",
|
||||
" {'generated_text': \"Synonyms of a word cat: big cat. but how come, we didn't hear it\"},\n",
|
||||
" {'generated_text': 'Synonyms of a word cat: \" mea - o - c \" which makes them sound'}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generator(\"Synonyms of a word cat:\", max_length=20, num_return_sequences=5)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> positive this is so horrible - > positive that your brother is gay - >'},\n",
|
||||
" {'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> negative i will bring this on you -, < positive am i, i'},\n",
|
||||
" {'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> negative i have self - esteem i must take it - : \\n - -'},\n",
|
||||
" {'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> negative this is - : \\n if it were true that the devil would have'},\n",
|
||||
" {'generated_text': \"I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> positive i have you - > positive it's a bad thing, > positive\"}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generator(\"I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this ->\", max_length=40, num_return_sequences=5)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'generated_text': 'Translate English to French: cat => chat, dog => chien, student => new and unusual. there were no more words to be'},\n",
|
||||
" {'generated_text': 'Translate English to French: cat => chat, dog => chien, student => student \\n his eyes were huge in his lean face as'},\n",
|
||||
" {'generated_text': \"Translate English to French: cat => chat, dog => chien, student => the teacher's words, their words, their words.\"}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generator(\"Translate English to French: cat => chat, dog => chien, student => \", top_k=50, max_length=30, num_return_sequences=3)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'generated_text': 'People who liked the movie The Matrix also liked it, and there was the movie of the first man after us. \\n i wanted to laugh at how stupid these stupid actors were. no, they were'},\n",
|
||||
" {'generated_text': \"People who liked the movie The Matrix also liked the movie, and the film was the result. and that's when the man in the story was brought into reality, after a few decades. \\n a\"},\n",
|
||||
" {'generated_text': 'People who liked the movie The Matrix also liked the movie the matrix, because there was a very old movie movie called the matrix, where there was a great super hero, and the super hero came out'},\n",
|
||||
" {'generated_text': \"People who liked the movie The Matrix also liked the movie that didn't have a chance to pay cash, if they could afford it. most often they got a good deal and a lot of money,\"},\n",
|
||||
" {'generated_text': \"People who liked the movie The Matrix also liked the movie, and i didn't seem to have the same problem. \\n i 'd met the other half of my family. i spent most of my time\"}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generator(\"People who liked the movie The Matrix also liked \", max_length=40, num_return_sequences=5)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 文本采样策略\n",
|
||||
"\n",
|
||||
"到目前为止，我们一直在使用简单的**贪婪**采样策略，即根据最高概率选择下一个词。其工作原理如下：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw my friend, a young man, sprawled across the bed in his bed. \\n \" hi, i\\'m mike eptirard. \" \\n there was silence on the other side of the door. i listened for any trace of life but there was nothing. my heart began to pound, i was starting to sweat, i took out my wallet'},\n",
|
||||
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw my mother on the bed, hugging her legs to her chest and sobbing. i saw my dad and mother from the corner of my eye. \\n elfin face was covered in tears as i entered the room. my dad and mother also wept ; just as they did every other time i came to work. but this time, they had different faces'},\n",
|
||||
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw the room had changed because it was dark. it still smelled like a hospital. a new light shined through from a vent in the ceiling. i found myself in a bathroom and a small room with a sink and a wall of glass. the bathroom billion years ago. not so different from all of the rest of the apartment. \\n now...'},\n",
|
||||
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a large woman with dark hair and pale skin. she was asleep, but i noticed a faint movement of her face. i could sense she was awake. i got up and walked over to her. \\n \" hello miss. i am inspector michael o\\'dell ; we are investigating the case against you. i wanted to ask if you were the'},\n",
|
||||
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw i had an empty table and three empty chairs. that was all i needed. i had left a note on a table in the center of the room and had a pen in hand. \" \\n \" i think what you were doing was something he was doing to her. \" \\n \" yeah, \" i nodded with a grin. \" i'}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"prompt = \"It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw\"\n",
|
||||
"generator(prompt,max_length=100,num_return_sequences=5)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**束搜索**允许生成器探索多个文本生成方向（*束*），并选择总体得分较高的方向。您可以通过提供`num_beams`参数来进行束搜索。您还可以指定`no_repeat_ngram_size`参数，以惩罚模型重复给定大小的n-gram：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting in a chair with his head in his hands. he didn\\'t look up as i approached. \\n \" excuse me, sir, \" i said. \" can i help you? \" \\n the man looked up at me. his eyes were red - rimmed and his face was pale, as if he hadn\\'t slept in days'},\n",
|
||||
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting at a desk in the middle of the room. he had his back to me, so i couldn\\'t see what he was doing. \" \\n \" what did he look like? \" i asked as i sat down on the bed next to her. \\n she took a deep breath and looked at me with tears in her eyes'},\n",
|
||||
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a woman sitting on the bed, reading a book. she looked up at me and smiled. \\n \" hi, \" she said. \" can i help you? \" \\n i sat down next to her and looked around the room. the walls were white, and there was a large window in the middle of the wall that looked out on'},\n",
|
||||
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting at a table in the middle of the room. he looked up as i walked in, and when he saw me, he got up and walked over to me. \\n \" can i help you? \" he asked as he put his hand on the small of my back and led me to a chair at the other end of'},\n",
|
||||
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a woman sitting on the edge of her bed, reading a book. she looked up at me and smiled. \\n \" hello, \" she said. \" can i help you? \" \\n i didn\\'t know what to say, so i just sat down in the chair next to the bed and looked at her. her hair was dark brown'}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"prompt = \"It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw\"\n",
|
||||
"generator(prompt,max_length=100,num_return_sequences=5,num_beams=10,no_repeat_ngram_size=2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**采样**使用模型返回的概率分布以非确定性方式选择下一个词。您可以通过参数`do_sample=True`启用采样。您还可以指定`temperature`，以使模型变得更或更少确定性。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw her. she was on the bed, but she looked very different. \\n \" honey, what\\'s the matter? \" i asked. \\n she sat up. \" i can\\'t believe it\\'s real. i\\'ve been dreaming about you for the last two days. \" \\n \" i can\\'t believe it either. i guess that\\'s how'}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"prompt = \"It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw\"\n",
|
||||
"generator(prompt,max_length=100,do_sample=True,temperature=0.8)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们还可以为采样提供额外的参数： \n",
|
||||
"* `top_k` 指定在使用采样时需要考虑的单词选项数量。这可以减少在文本中出现奇怪（低概率）单词的可能性。 \n",
|
||||
"* `top_p` 类似，但我们选择的是总概率大于 p 的最小高概率单词子集。 \n",
|
||||
"\n",
|
||||
"可以随意尝试添加这些参数进行实验。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 微调您的模型\n",
|
||||
"\n",
|
||||
"您还可以使用自己的数据集[微调您的模型](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/fine-tuning?pivots=programming-language-studio?WT.mc_id=academic-77998-bethanycheum)。这样可以在保留语言模型主要部分的同时，调整文本的风格。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.12 ('py38')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.11"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"coopTranslator": {
|
||||
"original_hash": "d4ff89615d38924a55594f16d6d20678",
|
||||
"translation_date": "2025-08-31T10:44:06+00:00",
|
||||
"source_file": "lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
# 预训练大型语言模型
|
||||
|
||||
在我们之前的所有任务中，我们都使用带标签的数据集训练神经网络来完成特定任务。而对于像 BERT 这样的大型 Transformer 模型，我们通过自监督的方式进行语言建模来构建语言模型，然后通过进一步的领域特定训练使其专注于特定的下游任务。然而，研究表明，大型语言模型也可以在没有任何领域特定训练的情况下解决许多任务。这类能够实现这一目标的模型家族被称为 **GPT**：生成式预训练 Transformer。
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/39)
|
||||
|
||||
## 文本生成与困惑度
|
||||
|
||||
神经网络能够在没有下游训练的情况下完成通用任务的理念在论文 [Language Models are Unsupervised Multitask Learners](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) 中提出。其核心思想是许多其他任务可以通过 **文本生成** 来建模，因为理解文本本质上意味着能够生成文本。由于模型在包含人类知识的大量文本上进行了训练，它也因此对广泛的主题有了深入的了解。
|
||||
|
||||
> 理解并能够生成文本也意味着对周围世界有所了解。人类在很大程度上也是通过阅读来学习的，而 GPT 网络在这方面与人类类似。
|
||||
|
||||
文本生成网络通过预测下一个词的概率 $$P(w_N)$$ 来工作。然而，下一个词的无条件概率等于该词在文本语料库中的出现频率。GPT 能够提供下一个词的 **条件概率**，即基于前面的词：$$P(w_N | w_{n-1}, ..., w_0)$$
|
||||
|
||||
> 您可以在我们的 [数据科学初学者课程](https://github.com/microsoft/Data-Science-For-Beginners/tree/main/1-Introduction/04-stats-and-probability) 中了解更多关于概率的内容。
|
||||
|
||||
语言生成模型的质量可以通过 **困惑度** 来定义。这是一种内在指标，可以在没有任何任务特定数据集的情况下衡量模型质量。它基于 *句子的概率* 的概念——模型对可能是真实的句子赋予高概率（即模型对其不 **困惑**），而对不太合理的句子（例如 *Can it does what?*）赋予低概率。当我们给模型提供真实文本语料库中的句子时，我们希望它们具有高概率和低 **困惑度**。数学上，困惑度定义为测试集的归一化逆概率：
|
||||
$$
|
||||
\mathrm{Perplexity}(W) = \sqrt[N]{1\over P(W_1,...,W_N)}
|
||||
$$
|
||||
|
||||
**您可以使用 [Hugging Face 的 GPT 驱动文本编辑器](https://transformer.huggingface.co/doc/gpt2-large) 进行文本生成实验**。在这个编辑器中，您开始编写文本，按下 **[TAB]** 键后会提供几个补全选项。如果选项太短或您不满意，可以再次按下 [TAB] 键，您将获得更多选项，包括更长的文本片段。
|
||||
|
||||
## GPT 是一个家族
|
||||
|
||||
GPT 不是单一模型，而是由 [OpenAI](https://openai.com) 开发和训练的一系列模型。
|
||||
|
||||
在 GPT 模型家族中，我们有：
|
||||
|
||||
| [GPT-2](https://huggingface.co/docs/transformers/model_doc/gpt2#openai-gpt2) | [GPT-3](https://openai.com/research/language-models-are-few-shot-learners) | [GPT-4](https://openai.com/gpt-4) |
|
||||
| -- | -- | -- |
|
||||
|语言模型，参数量高达 15 亿。| 语言模型，参数量高达 1750 亿。| 参数量达 100 万亿，支持图像和文本输入，并输出文本。|
|
||||
|
||||
GPT-3 和 GPT-4 模型可以通过 [Microsoft Azure 的认知服务](https://azure.microsoft.com/en-us/services/cognitive-services/openai-service/#overview?WT.mc_id=academic-77998-cacaste) 和 [OpenAI API](https://openai.com/api/) 使用。
|
||||
|
||||
## 提示工程
|
||||
|
||||
由于 GPT 在大量数据上进行了训练以理解语言和代码，它能够根据输入（提示）生成输出。提示是 GPT 的输入或查询，用户通过提示向模型提供任务指令以完成后续任务。为了获得期望的结果，您需要最有效的提示，这涉及选择合适的词语、格式、短语甚至符号。这种方法被称为 [提示工程](https://learn.microsoft.com/en-us/shows/ai-show/the-basics-of-prompt-engineering-with-azure-openai-service?WT.mc_id=academic-77998-bethanycheum)。
|
||||
|
||||
[此文档](https://learn.microsoft.com/en-us/semantic-kernel/prompt-engineering/?WT.mc_id=academic-77998-bethanycheum) 提供了有关提示工程的更多信息。
|
||||
|
||||
## ✍️ 示例笔记本：[使用 OpenAI-GPT 进行实验](GPT-PyTorch.ipynb)
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
* [使用 OpenAI-GPT 和 Hugging Face Transformers 生成文本](GPT-PyTorch.ipynb)
|
||||
|
||||
## 结论
|
||||
|
||||
新的通用预训练语言模型不仅能够建模语言结构，还包含大量自然语言知识。因此，它们可以在零样本或少样本设置中有效地解决一些 NLP 任务。
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/40)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
# 自然语言处理
|
||||
|
||||
|
||||
|
||||
在本节中，我们将重点使用神经网络来处理与**自然语言处理 (NLP)**相关的任务。我们希望计算机能够解决许多NLP问题：
|
||||
|
||||
* **文本分类**是一个典型的与文本序列相关的分类问题。例如，将电子邮件分类为垃圾邮件或非垃圾邮件，或者将文章分类为体育、商业、政治等类别。此外，在开发聊天机器人时，我们通常需要理解用户的意图——这就是所谓的**意图分类**。意图分类通常需要处理许多类别。
|
||||
* **情感分析**是一个典型的回归问题，我们需要为句子的正面/负面意义赋予一个数值（情感）。更高级的情感分析是**基于方面的情感分析** (ABSA)，它不是为整个句子赋予情感，而是为句子的不同部分（方面）赋予情感，例如：*在这家餐厅，我喜欢菜品，但氛围很糟糕*。
|
||||
* **命名实体识别** (NER) 是从文本中提取特定实体的问题。例如，我们需要理解在短语*我明天要飞往巴黎*中，*明天*指的是日期，*巴黎*是一个地点。
|
||||
* **关键词提取**与NER类似，但我们需要自动提取对句子意义重要的词，而无需针对特定实体类型进行预训练。
|
||||
* **文本聚类**在我们希望将相似的句子分组时非常有用，例如在技术支持对话中将相似的请求归类。
|
||||
* **问答**是指模型回答特定问题的能力。模型接收一个文本段落和一个问题作为输入，并需要提供文本中包含问题答案的位置（有时需要生成答案文本）。
|
||||
* **文本生成**是模型生成新文本的能力。它可以被视为一个分类任务，根据某些*文本提示*预测下一个字母/单词。高级文本生成模型（如GPT-3）能够通过一种称为[提示编程](https://towardsdatascience.com/software-3-0-how-prompting-will-change-the-rules-of-the-game-a982fbfe1e0)或[提示工程](https://medium.com/swlh/openai-gpt-3-and-prompt-engineering-dcdc2c5fcd29)的技术解决其他NLP任务，例如分类。
|
||||
* **文本摘要**是一种技术，我们希望计算机能够“阅读”长文本并用几句话进行总结。
|
||||
* **机器翻译**可以被视为一种结合一种语言的文本理解和另一种语言的文本生成的任务。
|
||||
|
||||
最初，大多数NLP任务是通过传统方法（如语法）解决的。例如，在机器翻译中，解析器用于将初始句子转换为语法树，然后提取高级语义结构以表示句子的意义，基于这种意义和目标语言的语法生成结果。如今，许多NLP任务通过神经网络更有效地解决。
|
||||
|
||||
> 许多经典的NLP方法在[自然语言处理工具包 (NLTK)](https://www.nltk.org) Python库中实现。在线有一本很棒的[NLTK书籍](https://www.nltk.org/book/)，涵盖了如何使用NLTK解决不同的NLP任务。
|
||||
|
||||
在我们的课程中，我们将主要关注使用神经网络进行NLP，并在需要时使用NLTK。
|
||||
|
||||
我们已经学习了如何使用神经网络处理表格数据和图像。这些数据类型与文本的主要区别在于，文本是一个可变长度的序列，而图像的输入大小是预先确定的。虽然卷积网络可以从输入数据中提取模式，但文本中的模式更复杂。例如，否定可能与主语分隔许多单词（例如：*我不喜欢橙子*，与*我不喜欢那些又大又多彩又美味的橙子*），但仍应被解释为一个模式。因此，为了处理语言，我们需要引入新的神经网络类型，例如*循环网络*和*Transformer*。
|
||||
|
||||
## 安装库
|
||||
|
||||
如果您使用本地Python安装运行本课程，可能需要使用以下命令安装所有NLP所需的库：
|
||||
|
||||
**对于PyTorch**
|
||||
```bash
|
||||
pip install -r requirements-torch.txt
|
||||
```
|
||||
**对于TensorFlow**
|
||||
```bash
|
||||
pip install -r requirements-tf.txt
|
||||
```
|
||||
|
||||
> 您可以在[Microsoft Learn](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-tensorflow/?WT.mc_id=academic-77998-cacaste)上尝试使用TensorFlow进行NLP。
|
||||
|
||||
## GPU警告
|
||||
|
||||
在本节中，我们将在一些示例中训练较大的模型。
|
||||
* **使用支持GPU的计算机**：建议在支持GPU的计算机上运行您的笔记本，以减少训练大型模型时的等待时间。
|
||||
* **GPU内存限制**：在GPU上运行可能会导致GPU内存不足的情况，尤其是在训练大型模型时。
|
||||
* **GPU内存消耗**：训练期间消耗的GPU内存量取决于多个因素，包括小批量大小。
|
||||
* **最小化小批量大小**：如果遇到GPU内存问题，可以考虑减少代码中的小批量大小作为解决方案。
|
||||
* **TensorFlow GPU内存释放**：旧版本的TensorFlow可能无法正确释放GPU内存，尤其是在一个Python内核中训练多个模型时。为了有效管理GPU内存使用，可以配置TensorFlow仅在需要时分配GPU内存。
|
||||
* **代码包含**：要设置TensorFlow仅在需要时增长GPU内存分配，请在笔记本中包含以下代码：
|
||||
|
||||
```python
|
||||
physical_devices = tf.config.list_physical_devices('GPU')
|
||||
if len(physical_devices)>0:
|
||||
tf.config.experimental.set_memory_growth(physical_devices[0], True)
|
||||
```
|
||||
|
||||
如果您对从经典机器学习角度学习NLP感兴趣，请访问[这套课程](https://github.com/microsoft/ML-For-Beginners/tree/main/6-NLP)。
|
||||
|
||||
## 本节内容
|
||||
在本节中，我们将学习：
|
||||
|
||||
* [将文本表示为张量](13-TextRep/README.md)
|
||||
* [词嵌入](14-Emdeddings/README.md)
|
||||
* [语言建模](15-LanguageModeling/README.md)
|
||||
* [循环神经网络](16-RNN/README.md)
|
||||
* [生成网络](17-GenerativeNetworks/README.md)
|
||||
* [Transformer](18-Transformers/README.md)
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 作业：丢番图方程\n",
|
||||
"\n",
|
||||
"> 本作业是 [AI for Beginners Curriculum](http://github.com/microsoft/ai-for-beginners) 的一部分，并受到 [这篇文章](https://habr.com/post/128704/) 的启发。\n",
|
||||
"\n",
|
||||
"你的目标是解决所谓的 **丢番图方程**——一个具有整数根和整数系数的方程。例如，考虑以下方程：\n",
|
||||
"\n",
|
||||
"$$a+2b+3c+4d=30$$\n",
|
||||
"\n",
|
||||
"你需要找到满足该方程的整数根 $a$,$b$,$c$,$d\\in\\mathbb{N}$。\n",
|
||||
"\n",
|
||||
"提示：\n",
|
||||
"1. 可以考虑根值在区间 [0;30] 内\n",
|
||||
"1. 作为基因，可以考虑使用根值列表\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。原始语言的文档应被视为权威来源。对于关键信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"coopTranslator": {
|
||||
"original_hash": "a967e1fa1e11ab2b6467b19349a4a9aa",
|
||||
"translation_date": "2025-08-31T09:21:13+00:00",
|
||||
"source_file": "lessons/6-Other/21-GeneticAlgorithms/Diophantine.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,80 @@
|
|||
# 遗传算法
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/41)
|
||||
|
||||
**遗传算法** (GA) 基于一种**进化式方法**，通过模拟种群的进化过程来为给定问题找到最优解。这种方法由 [John Henry Holland](https://wikipedia.org/wiki/John_Henry_Holland) 于1975年提出。
|
||||
|
||||
遗传算法基于以下理念：
|
||||
|
||||
* 问题的有效解可以表示为**基因**
|
||||
* **交叉**允许我们将两个解结合起来，生成一个新的有效解
|
||||
* 使用某种**适应度函数**进行**选择**，以筛选出更优的解
|
||||
* 引入**变异**以打破优化过程的稳定性，从而摆脱局部最小值
|
||||
|
||||
如果你想实现遗传算法，需要以下步骤：
|
||||
|
||||
* 找到一种方法，用**基因** g∈Γ 编码问题的解
|
||||
* 在基因集合 Γ 上定义**适应度函数** fit: Γ→**R**，函数值越小表示解越优。
|
||||
* 定义**交叉**机制，将两个基因结合生成一个新的有效解 crossover: Γ<sup>2</sub>→Γ。
|
||||
* 定义**变异**机制 mutate: Γ→Γ。
|
||||
|
||||
在许多情况下，交叉和变异是操作基因的简单算法，比如处理数字序列或位向量。
|
||||
|
||||
遗传算法的具体实现可能因情况而异，但总体结构如下：
|
||||
|
||||
1. 选择初始种群 G⊂Γ
|
||||
2. 随机选择将在此步骤执行的操作：交叉或变异
|
||||
3. **交叉**：
|
||||
* 随机选择两个基因 g<sub>1</sub>, g<sub>2</sub> ∈ G
|
||||
* 计算交叉结果 g=crossover(g<sub>1</sub>,g<sub>2</sub>)
|
||||
* 如果 fit(g)<fit(g<sub>1</sub>) 或 fit(g)<fit(g<sub>2</sub>)，用 g 替换种群中的对应基因。
|
||||
4. **变异** - 随机选择一个基因 g∈G，并用 mutate(g) 替换它
|
||||
5. 从步骤2开始重复，直到适应度函数值足够小，或达到步骤数限制。
|
||||
|
||||
## 常见任务
|
||||
|
||||
遗传算法通常解决以下任务：
|
||||
|
||||
1. 排程优化
|
||||
1. 最优装箱
|
||||
1. 最优切割
|
||||
1. 加速穷举搜索
|
||||
|
||||
## ✍️ 练习：遗传算法
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
访问[这个笔记本](Genetic.ipynb)，查看两个使用遗传算法的示例：
|
||||
|
||||
1. 公平分配宝藏
|
||||
1. 八皇后问题
|
||||
|
||||
## 总结
|
||||
|
||||
遗传算法被用于解决许多问题，包括物流和搜索问题。这个领域的灵感来源于心理学和计算机科学的交叉研究。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
“遗传算法易于实现，但其行为难以理解。” [来源](https://wikipedia.org/wiki/Genetic_algorithm) 进行一些研究，找到一个遗传算法的实现，例如解决数独问题，并以草图或流程图的形式解释其工作原理。
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/42)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
观看[这个精彩视频](https://www.youtube.com/watch?v=qv6UVOQ0F44)，了解计算机如何通过遗传算法训练的神经网络学习玩超级马里奥。我们将在[下一节](../22-DeepRL/README.md)中学习更多关于计算机学习玩游戏的内容。
|
||||
|
||||
## [作业：丢番图方程](Diophantine.ipynb)
|
||||
|
||||
你的目标是解决所谓的**丢番图方程**——一个具有整数根的方程。例如，考虑方程 a+2b+3c+4d=30。你需要找到满足该方程的整数根。
|
||||
|
||||
*此作业的灵感来源于[这篇文章](https://habr.com/post/128704/)。*
|
||||
|
||||
提示：
|
||||
|
||||
1. 可以考虑根的范围为 [0;30]
|
||||
1. 作为基因，可以使用根值列表
|
||||
|
||||
使用 [Diophantine.ipynb](Diophantine.ipynb) 作为起点。
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,501 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 使用强化学习训练平衡摆杆\n",
|
||||
"\n",
|
||||
"本笔记本是 [AI for Beginners Curriculum](http://aka.ms/ai-beginners) 的一部分，灵感来源于 [官方 PyTorch 教程](https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html) 和 [这个 Cartpole PyTorch 实现](https://github.com/yc930401/Actor-Critic-pytorch)。\n",
|
||||
"\n",
|
||||
"在这个示例中，我们将使用强化学习（RL）训练一个模型，使其能够在水平轨道上移动的推车上平衡摆杆。我们将使用 [OpenAI Gym](https://www.gymlibrary.ml/) 环境来模拟摆杆。\n",
|
||||
"\n",
|
||||
"> **注意**: 你可以在本地运行本课程的代码（例如，通过 Visual Studio Code），此时模拟器会在新窗口中打开。如果在线运行代码，你可能需要对代码进行一些调整，具体说明见 [这里](https://towardsdatascience.com/rendering-openai-gym-envs-on-binder-and-google-colab-536f99391cc7)。\n",
|
||||
"\n",
|
||||
"我们将从确保 Gym 已安装开始：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"!{sys.executable} -m pip install gym"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在让我们创建 CartPole 环境，并了解如何对其进行操作。一个环境具有以下属性：\n",
|
||||
"\n",
|
||||
"* **动作空间** 是我们在模拟的每一步中可以执行的所有可能动作的集合 \n",
|
||||
"* **观察空间** 是我们能够获取的所有可能观察值的集合 \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import gym\n",
|
||||
"\n",
|
||||
"env = gym.make(\"CartPole-v1\")\n",
|
||||
"\n",
|
||||
"print(f\"Action space: {env.action_space}\")\n",
|
||||
"print(f\"Observation space: {env.observation_space}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"让我们看看模拟是如何运行的。以下循环会运行模拟，直到 `env.step` 不返回终止标志 `done`。我们将使用 `env.action_space.sample()` 随机选择动作，这意味着实验可能会非常快地失败（当小车的速度、位置或角度超出某些限制时，CartPole 环境会终止）。\n",
|
||||
"\n",
|
||||
"> 模拟会在新窗口中打开。你可以多次运行代码，观察它的表现。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"env.reset()\n",
|
||||
"\n",
|
||||
"done = False\n",
|
||||
"total_reward = 0\n",
|
||||
"while not done:\n",
|
||||
" env.render()\n",
|
||||
" obs, rew, done, info = env.step(env.action_space.sample())\n",
|
||||
" total_reward += rew\n",
|
||||
" print(f\"{obs} -> {rew}\")\n",
|
||||
"print(f\"Total reward: {total_reward}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"你会注意到观测值包含4个数字，它们是：\n",
|
||||
"- 小车的位置\n",
|
||||
"- 小车的速度\n",
|
||||
"- 杆子的角度\n",
|
||||
"- 杆子的旋转速度\n",
|
||||
"\n",
|
||||
"`rew` 是我们在每一步中获得的奖励。在 CartPole 环境中，每进行一步模拟都会获得1分奖励，目标是最大化总奖励，也就是让小车平衡杆子而不倒下的时间尽可能长。\n",
|
||||
"\n",
|
||||
"在强化学习中，我们的目标是训练一个**策略** $\\pi$，它会根据每个状态 $s$ 告诉我们应该采取的动作 $a$，本质上就是 $a = \\pi(s)$。\n",
|
||||
"\n",
|
||||
"如果你想要一个概率性的解决方案，可以将策略视为为每个动作返回一组概率，也就是说 $\\pi(a|s)$ 表示在状态 $s$ 下采取动作 $a$ 的概率。\n",
|
||||
"\n",
|
||||
"## 策略梯度方法\n",
|
||||
"\n",
|
||||
"在最简单的强化学习算法中，称为**策略梯度**，我们将训练一个神经网络来预测下一步的动作。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import torch\n",
|
||||
"\n",
|
||||
"num_inputs = 4\n",
|
||||
"num_actions = 2\n",
|
||||
"\n",
|
||||
"model = torch.nn.Sequential(\n",
|
||||
" torch.nn.Linear(num_inputs, 128, bias=False, dtype=torch.float32),\n",
|
||||
" torch.nn.ReLU(),\n",
|
||||
" torch.nn.Linear(128, num_actions, bias = False, dtype=torch.float32),\n",
|
||||
" torch.nn.Softmax(dim=1)\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们将通过运行许多实验来训练网络，并在每次运行后更新我们的网络。让我们定义一个函数来运行实验并返回结果（所谓的**轨迹**）——所有状态、动作（及其推荐概率）和奖励：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def run_episode(max_steps_per_episode = 10000,render=False): \n",
|
||||
" states, actions, probs, rewards = [],[],[],[]\n",
|
||||
" state = env.reset()\n",
|
||||
" for _ in range(max_steps_per_episode):\n",
|
||||
" if render:\n",
|
||||
" env.render()\n",
|
||||
" action_probs = model(torch.from_numpy(np.expand_dims(state,0)))[0]\n",
|
||||
" action = np.random.choice(num_actions, p=np.squeeze(action_probs.detach().numpy()))\n",
|
||||
" nstate, reward, done, info = env.step(action)\n",
|
||||
" if done:\n",
|
||||
" break\n",
|
||||
" states.append(state)\n",
|
||||
" actions.append(action)\n",
|
||||
" probs.append(action_probs.detach().numpy())\n",
|
||||
" rewards.append(reward)\n",
|
||||
" state = nstate\n",
|
||||
" return np.vstack(states), np.vstack(actions), np.vstack(probs), np.vstack(rewards)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"您可以运行一个未训练网络的情节，并观察到总奖励（即情节长度）非常低：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"s, a, p, r = run_episode()\n",
|
||||
"print(f\"Total reward: {np.sum(r)}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"政策梯度算法的一个棘手方面是使用**折扣奖励**。其思想是我们在游戏的每一步计算总奖励的向量，并在此过程中使用某个系数 $gamma$ 对早期奖励进行折扣。我们还对结果向量进行归一化，因为我们将使用它作为权重来影响我们的训练：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"eps = 0.0001\n",
|
||||
"\n",
|
||||
"def discounted_rewards(rewards,gamma=0.99,normalize=True):\n",
|
||||
" ret = []\n",
|
||||
" s = 0\n",
|
||||
" for r in rewards[::-1]:\n",
|
||||
" s = r + gamma * s\n",
|
||||
" ret.insert(0, s)\n",
|
||||
" if normalize:\n",
|
||||
" ret = (ret-np.mean(ret))/(np.std(ret)+eps)\n",
|
||||
" return ret"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在开始实际训练！我们将运行300次实验，每次实验中我们将执行以下步骤：\n",
|
||||
"\n",
|
||||
"1. 运行实验并收集轨迹数据。\n",
|
||||
"2. 计算所采取的动作与预测概率之间的差异（`gradients`）。差异越小，说明我们越确信采取了正确的动作。\n",
|
||||
"3. 计算折扣奖励，并将梯度乘以折扣奖励——这样可以确保高奖励的步骤对最终结果的影响比低奖励的步骤更大。\n",
|
||||
"4. 我们的神经网络的目标动作部分来自运行期间的预测概率，部分来自计算出的梯度。我们将使用`alpha`参数来确定梯度和奖励在多大程度上被考虑——这被称为强化学习算法的*学习率*。\n",
|
||||
"5. 最后，我们基于状态和目标动作训练网络，并重复这一过程。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"optimizer = torch.optim.Adam(model.parameters(), lr=0.01)\n",
|
||||
"\n",
|
||||
"def train_on_batch(x, y):\n",
|
||||
" x = torch.from_numpy(x)\n",
|
||||
" y = torch.from_numpy(y)\n",
|
||||
" optimizer.zero_grad()\n",
|
||||
" predictions = model(x)\n",
|
||||
" loss = -torch.mean(torch.log(predictions) * y)\n",
|
||||
" loss.backward()\n",
|
||||
" optimizer.step()\n",
|
||||
" return loss"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"alpha = 1e-4\n",
|
||||
"\n",
|
||||
"history = []\n",
|
||||
"for epoch in range(300):\n",
|
||||
" states, actions, probs, rewards = run_episode()\n",
|
||||
" one_hot_actions = np.eye(2)[actions.T][0]\n",
|
||||
" gradients = one_hot_actions-probs\n",
|
||||
" dr = discounted_rewards(rewards)\n",
|
||||
" gradients *= dr\n",
|
||||
" target = alpha*np.vstack([gradients])+probs\n",
|
||||
" train_on_batch(states,target)\n",
|
||||
" history.append(np.sum(rewards))\n",
|
||||
" if epoch%100==0:\n",
|
||||
" print(f\"{epoch} -> {np.sum(rewards)}\")\n",
|
||||
"\n",
|
||||
"plt.plot(history)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在让我们运行带有渲染的剧集来查看结果：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"_ = run_episode(render=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"希望你能看到，现在杆子已经可以很好地保持平衡了！\n",
|
||||
"\n",
|
||||
"## Actor-Critic 模型\n",
|
||||
"\n",
|
||||
"Actor-Critic 模型是策略梯度的进一步发展，在该模型中，我们构建一个神经网络来同时学习策略和估计奖励。这个网络将有两个输出（或者你可以将其视为两个独立的网络）：\n",
|
||||
"* **Actor** 将通过给出状态概率分布来推荐采取的行动，就像在策略梯度模型中一样。\n",
|
||||
"* **Critic** 将估算这些行动可能带来的奖励。它会返回在给定状态下未来的总估计奖励。\n",
|
||||
"\n",
|
||||
"让我们定义这样一个模型：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from itertools import count\n",
|
||||
"import torch.nn.functional as F"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
|
||||
"env = gym.make(\"CartPole-v1\")\n",
|
||||
"\n",
|
||||
"state_size = env.observation_space.shape[0]\n",
|
||||
"action_size = env.action_space.n\n",
|
||||
"lr = 0.0001\n",
|
||||
"\n",
|
||||
"class Actor(torch.nn.Module):\n",
|
||||
" def __init__(self, state_size, action_size):\n",
|
||||
" super(Actor, self).__init__()\n",
|
||||
" self.state_size = state_size\n",
|
||||
" self.action_size = action_size\n",
|
||||
" self.linear1 = torch.nn.Linear(self.state_size, 128)\n",
|
||||
" self.linear2 = torch.nn.Linear(128, 256)\n",
|
||||
" self.linear3 = torch.nn.Linear(256, self.action_size)\n",
|
||||
"\n",
|
||||
" def forward(self, state):\n",
|
||||
" output = F.relu(self.linear1(state))\n",
|
||||
" output = F.relu(self.linear2(output))\n",
|
||||
" output = self.linear3(output)\n",
|
||||
" distribution = torch.distributions.Categorical(F.softmax(output, dim=-1))\n",
|
||||
" return distribution\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class Critic(torch.nn.Module):\n",
|
||||
" def __init__(self, state_size, action_size):\n",
|
||||
" super(Critic, self).__init__()\n",
|
||||
" self.state_size = state_size\n",
|
||||
" self.action_size = action_size\n",
|
||||
" self.linear1 = torch.nn.Linear(self.state_size, 128)\n",
|
||||
" self.linear2 = torch.nn.Linear(128, 256)\n",
|
||||
" self.linear3 = torch.nn.Linear(256, 1)\n",
|
||||
"\n",
|
||||
" def forward(self, state):\n",
|
||||
" output = F.relu(self.linear1(state))\n",
|
||||
" output = F.relu(self.linear2(output))\n",
|
||||
" value = self.linear3(output)\n",
|
||||
" return value"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们需要稍微修改我们的 `discounted_rewards` 和 `run_episode` 函数：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def discounted_rewards(next_value, rewards, masks, gamma=0.99):\n",
|
||||
" R = next_value\n",
|
||||
" returns = []\n",
|
||||
" for step in reversed(range(len(rewards))):\n",
|
||||
" R = rewards[step] + gamma * R * masks[step]\n",
|
||||
" returns.insert(0, R)\n",
|
||||
" return returns\n",
|
||||
"\n",
|
||||
"def run_episode(actor, critic, n_iters):\n",
|
||||
" optimizerA = torch.optim.Adam(actor.parameters())\n",
|
||||
" optimizerC = torch.optim.Adam(critic.parameters())\n",
|
||||
" for iter in range(n_iters):\n",
|
||||
" state = env.reset()\n",
|
||||
" log_probs = []\n",
|
||||
" values = []\n",
|
||||
" rewards = []\n",
|
||||
" masks = []\n",
|
||||
" entropy = 0\n",
|
||||
" env.reset()\n",
|
||||
"\n",
|
||||
" for i in count():\n",
|
||||
" env.render()\n",
|
||||
" state = torch.FloatTensor(state).to(device)\n",
|
||||
" dist, value = actor(state), critic(state)\n",
|
||||
"\n",
|
||||
" action = dist.sample()\n",
|
||||
" next_state, reward, done, _ = env.step(action.cpu().numpy())\n",
|
||||
"\n",
|
||||
" log_prob = dist.log_prob(action).unsqueeze(0)\n",
|
||||
" entropy += dist.entropy().mean()\n",
|
||||
"\n",
|
||||
" log_probs.append(log_prob)\n",
|
||||
" values.append(value)\n",
|
||||
" rewards.append(torch.tensor([reward], dtype=torch.float, device=device))\n",
|
||||
" masks.append(torch.tensor([1-done], dtype=torch.float, device=device))\n",
|
||||
"\n",
|
||||
" state = next_state\n",
|
||||
"\n",
|
||||
" if done:\n",
|
||||
" print('Iteration: {}, Score: {}'.format(iter, i))\n",
|
||||
" break\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" next_state = torch.FloatTensor(next_state).to(device)\n",
|
||||
" next_value = critic(next_state)\n",
|
||||
" returns = discounted_rewards(next_value, rewards, masks)\n",
|
||||
"\n",
|
||||
" log_probs = torch.cat(log_probs)\n",
|
||||
" returns = torch.cat(returns).detach()\n",
|
||||
" values = torch.cat(values)\n",
|
||||
"\n",
|
||||
" advantage = returns - values\n",
|
||||
"\n",
|
||||
" actor_loss = -(log_probs * advantage.detach()).mean()\n",
|
||||
" critic_loss = advantage.pow(2).mean()\n",
|
||||
"\n",
|
||||
" optimizerA.zero_grad()\n",
|
||||
" optimizerC.zero_grad()\n",
|
||||
" actor_loss.backward()\n",
|
||||
" critic_loss.backward()\n",
|
||||
" optimizerA.step()\n",
|
||||
" optimizerC.step()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在我们将运行主要训练循环。我们将通过计算适当的损失函数并更新网络参数来使用手动网络训练过程：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\n",
|
||||
"actor = Actor(state_size, action_size).to(device)\n",
|
||||
"critic = Critic(state_size, action_size).to(device)\n",
|
||||
"run_episode(actor, critic, n_iters=100)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"env.close()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 要点\n",
|
||||
"\n",
|
||||
"在这个演示中，我们看到了两种强化学习算法：简单的策略梯度算法和更复杂的演员-评论家算法。可以看到，这些算法使用的是状态、动作和奖励的抽象概念，因此它们可以应用于非常不同的环境。\n",
|
||||
"\n",
|
||||
"强化学习使我们能够仅通过观察最终奖励来学习解决问题的最佳策略。不需要标注数据集这一点使我们可以多次重复模拟来优化模型。然而，强化学习领域仍然存在许多挑战，如果你决定更多地关注这个有趣的人工智能领域，你将会进一步了解这些挑战。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。原始语言的文档应被视为权威来源。对于重要信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.10.4 64-bit",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.4"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
|
||||
}
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "04f8d9978cd11281d81dd037cbf6ce20",
|
||||
"translation_date": "2025-08-31T09:26:08+00:00",
|
||||
"source_file": "lessons/6-Other/22-DeepRL/CartPole-RL-PyTorch.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,117 @@
|
|||
# 深度强化学习
|
||||
|
||||
强化学习（RL）被认为是机器学习的基本范式之一，与监督学习和无监督学习并列。在监督学习中，我们依赖于具有已知结果的数据集，而强化学习则基于**通过实践学习**。例如，当我们第一次看到一个电脑游戏时，即使不知道规则，我们也会开始玩，并通过玩游戏和调整行为逐渐提高自己的技能。
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/43)
|
||||
|
||||
要进行强化学习，我们需要：
|
||||
|
||||
* 一个**环境**或**模拟器**，它定义了游戏规则。我们应该能够在模拟器中运行实验并观察结果。
|
||||
* 某种**奖励函数**，它指示我们的实验是否成功。例如，在学习玩电脑游戏时，奖励可以是我们的最终得分。
|
||||
|
||||
基于奖励函数，我们应该能够调整自己的行为并提高技能，以便下一次表现更好。强化学习与其他类型的机器学习的主要区别在于，在强化学习中，我们通常不知道自己是否赢了或输了，直到游戏结束。因此，我们无法判断单独的某个动作是否是好的——我们只有在游戏结束时才会收到奖励。
|
||||
|
||||
在强化学习过程中，我们通常会进行许多实验。在每次实验中，我们需要在遵循迄今为止学到的最佳策略（**利用**）和探索新的可能状态（**探索**）之间找到平衡。
|
||||
|
||||
## OpenAI Gym
|
||||
|
||||
一个非常适合强化学习的工具是 [OpenAI Gym](https://gym.openai.com/)——一个**模拟环境**，它可以模拟许多不同的环境，从 Atari 游戏到物理学中的杆平衡问题。它是训练强化学习算法最流行的模拟环境之一，由 [OpenAI](https://openai.com/) 维护。
|
||||
|
||||
> **Note**: 你可以在 [这里](https://gym.openai.com/envs/#classic_control) 查看 OpenAI Gym 提供的所有环境。
|
||||
|
||||
## CartPole 平衡
|
||||
|
||||
你可能都见过现代平衡设备，比如 *Segway* 或 *Gyroscooters*。它们能够通过调整轮子来响应加速度计或陀螺仪的信号，从而自动保持平衡。在本节中，我们将学习如何解决一个类似的问题——杆平衡。这类似于马戏团表演者需要在手上平衡杆的情况——但这种杆平衡仅发生在一维空间中。
|
||||
|
||||
一种简化的平衡问题被称为 **CartPole** 问题。在 CartPole 世界中，我们有一个可以左右移动的水平滑块，目标是在滑块上方保持一个垂直杆的平衡。
|
||||
|
||||
<img alt="a cartpole" src="../../../../../translated_images/zh-CN/cartpole.f52a67f27e058170.webp" width="200"/>
|
||||
|
||||
要创建和使用这个环境，我们需要几行 Python 代码：
|
||||
|
||||
```python
|
||||
import gym
|
||||
env = gym.make("CartPole-v1")
|
||||
|
||||
env.reset()
|
||||
done = False
|
||||
total_reward = 0
|
||||
while not done:
|
||||
env.render()
|
||||
action = env.action_space.sample()
|
||||
observaton, reward, done, info = env.step(action)
|
||||
total_reward += reward
|
||||
|
||||
print(f"Total reward: {total_reward}")
|
||||
```
|
||||
|
||||
每个环境都可以通过以下方式访问：
|
||||
* `env.reset` 开始一个新的实验
|
||||
* `env.step` 执行一个模拟步骤。它接收来自**动作空间**的一个**动作**，并返回一个**观察值**（来自观察空间），以及奖励和终止标志。
|
||||
|
||||
在上面的示例中，我们在每一步执行随机动作，因此实验的持续时间非常短：
|
||||
|
||||
|
||||
|
||||
强化学习算法的目标是训练一个模型——所谓的**策略** π——它会根据给定状态返回动作。我们也可以将策略视为概率性的，例如，对于任何状态 *s* 和动作 *a*，它会返回我们在状态 *s* 下采取动作 *a* 的概率 π(*a*|*s*)。
|
||||
|
||||
## 策略梯度算法
|
||||
|
||||
建模策略的最明显方法是创建一个神经网络，该网络将状态作为输入，并返回相应的动作（或者更确切地说是所有动作的概率）。从某种意义上说，它类似于普通的分类任务，但有一个主要区别——我们事先不知道在每一步应该采取哪些动作。
|
||||
|
||||
这里的想法是估计这些概率。我们构建一个**累计奖励**向量，显示实验中每一步的总奖励。我们还通过乘以某个系数 γ=0.99 对早期奖励进行**奖励折扣**，以减小早期奖励的影响。然后，我们强化实验路径中产生较大奖励的步骤。
|
||||
|
||||
> 了解更多关于策略梯度算法的信息，并在[示例笔记本](CartPole-RL-TF.ipynb)中查看其实际应用。
|
||||
|
||||
## Actor-Critic 算法
|
||||
|
||||
策略梯度方法的改进版本被称为 **Actor-Critic**。其核心思想是神经网络将被训练以返回两件事：
|
||||
|
||||
* 策略，决定采取的动作。这部分称为 **actor**。
|
||||
* 对当前状态下可以获得的总奖励的估计——这部分称为 **critic**。
|
||||
|
||||
从某种意义上说，这种架构类似于 [GAN](../../4-ComputerVision/10-GANs/README.md)，其中有两个网络相互对抗进行训练。在 Actor-Critic 模型中，actor 提出我们需要采取的动作，而 critic 尝试进行批判并估计结果。然而，我们的目标是让这两个网络协同训练。
|
||||
|
||||
因为我们在实验过程中同时知道真实的累计奖励和 critic 返回的结果，因此构建一个损失函数以最小化它们之间的差异相对容易。这将产生**critic 损失**。我们可以使用与策略梯度算法相同的方法计算**actor 损失**。
|
||||
|
||||
运行这些算法后，我们可以期待我们的 CartPole 表现如下：
|
||||
|
||||
|
||||
|
||||
## ✍️ 练习：策略梯度和 Actor-Critic 强化学习
|
||||
|
||||
通过以下笔记本继续学习：
|
||||
|
||||
* [TensorFlow 中的强化学习](CartPole-RL-TF.ipynb)
|
||||
* [PyTorch 中的强化学习](CartPole-RL-PyTorch.ipynb)
|
||||
|
||||
## 其他强化学习任务
|
||||
|
||||
如今，强化学习是一个快速发展的研究领域。一些有趣的强化学习示例包括：
|
||||
|
||||
* 教电脑玩 **Atari 游戏**。这个问题的挑战在于我们没有简单的状态表示为向量，而是一个截图——我们需要使用 CNN 将屏幕图像转换为特征向量或提取奖励信息。Atari 游戏可以在 Gym 中找到。
|
||||
* 教电脑玩棋盘游戏，比如国际象棋和围棋。最近，像 **Alpha Zero** 这样的最先进程序通过两个代理相互对战并在每一步中不断改进，从零开始进行训练。
|
||||
* 在工业中，强化学习用于从模拟中创建控制系统。一个名为 [Bonsai](https://azure.microsoft.com/services/project-bonsai/?WT.mc_id=academic-77998-cacaste) 的服务专门为此设计。
|
||||
|
||||
## 总结
|
||||
|
||||
我们现在已经学习了如何通过提供定义游戏期望状态的奖励函数，并让代理智能地探索搜索空间，来训练代理以获得良好的结果。我们成功尝试了两种算法，并在相对较短的时间内取得了良好的结果。然而，这只是你强化学习旅程的开始，如果你想深入研究，应该考虑参加专门的课程。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
探索“其他强化学习任务”部分列出的应用，并尝试实现其中一个！
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/44)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
在我们的[初学者机器学习课程](https://github.com/microsoft/ML-For-Beginners/blob/main/8-Reinforcement/README.md)中了解更多关于经典强化学习的内容。
|
||||
|
||||
观看[这个精彩视频](https://www.youtube.com/watch?v=qv6UVOQ0F44)，了解电脑如何学习玩超级马里奥。
|
||||
|
||||
## 作业：[训练一个山地车](lab/README.md)
|
||||
|
||||
在这个作业中，你的目标是训练一个不同的 Gym 环境——[山地车](https://www.gymlibrary.ml/environments/classic_control/mountain_car/)。
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,109 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 训练山地车逃脱\n",
|
||||
"\n",
|
||||
"来自 [AI for Beginners Curriculum](https://github.com/microsoft/ai-for-beginners) 的实验任务。\n",
|
||||
"\n",
|
||||
"你的目标是训练强化学习代理来控制 OpenAI 环境中的 [Mountain Car](https://www.gymlibrary.ml/environments/classic_control/mountain_car/)。\n",
|
||||
"\n",
|
||||
"让我们从创建环境开始：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import gym\n",
|
||||
"env = gym.make('MountainCar-v0')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"让我们看看随机实验是什么样子的：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"state = env.reset()\n",
|
||||
"while True:\n",
|
||||
" env.render()\n",
|
||||
" action = env.action_space.sample()\n",
|
||||
" state, reward, done, info = env.step(action)\n",
|
||||
" if done:\n",
|
||||
" break"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"## Lost of code here"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"env.close()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.8.12 ('py38')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.12"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "f062b3b18449593ef8e0fcc029868781",
|
||||
"translation_date": "2025-08-31T09:28:54+00:00",
|
||||
"source_file": "lessons/6-Other/22-DeepRL/lab/MountainCar.ipynb",
|
||||
"language_code": "zh"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
## 环境
|
||||
|
||||
Mountain Car 环境中，汽车被困在一个山谷中。你的目标是跳出山谷并到达旗帜处。你可以执行的动作包括向左加速、向右加速或保持不动。你可以观察汽车在 x 轴上的位置以及速度。
|
||||
|
||||
## 起始笔记本
|
||||
|
||||
通过打开 [MountainCar.ipynb](../../../../../../lessons/6-Other/22-DeepRL/lab/MountainCar.ipynb) 开始实验。
|
||||
|
||||
## 收获
|
||||
|
||||
在整个实验过程中，你应该学到将 RL 算法应用到一个新环境通常是非常直接的，因为 OpenAI Gym 对所有环境都提供了统一的接口，而算法本身并不太依赖于环境的具体性质。你甚至可以重新组织 Python 代码，以便将任何环境作为参数传递给 RL 算法。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。因使用本翻译而引起的任何误解或误读，我们概不负责。
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,157 @@
|
|||
# 多智能体系统
|
||||
|
||||
实现智能的一种可能方法是所谓的**涌现**（或**协同**）方法，其基于这样一个事实：许多相对简单的智能体的组合行为可以导致整个系统表现出更复杂（或更智能）的行为。从理论上讲，这种方法基于[集体智能](https://en.wikipedia.org/wiki/Collective_intelligence)、[涌现论](https://en.wikipedia.org/wiki/Global_brain)和[进化控制论](https://en.wikipedia.org/wiki/Global_brain)的原则，这些理论认为，当低级系统被适当地组合成高级系统时，高级系统会获得某种附加价值（即所谓的*元系统转变原则*）。
|
||||
|
||||
## [课前测验](https://ff-quizzes.netlify.app/en/ai/quiz/45)
|
||||
|
||||
**多智能体系统**这一方向在1990年代随着互联网和分布式系统的发展在人工智能领域兴起。经典的人工智能教材之一[《人工智能：现代方法》](https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Modern_Approach)从多智能体系统的角度探讨了经典人工智能的观点。
|
||||
|
||||
多智能体方法的核心是**智能体**的概念——一个生活在某种**环境**中的实体，它可以感知环境并对其采取行动。这是一个非常广泛的定义，智能体可以有许多不同的类型和分类：
|
||||
|
||||
* 按其推理能力：
|
||||
- **反应型**智能体通常具有简单的请求-响应行为
|
||||
- **推理型**智能体采用某种逻辑推理和/或规划能力
|
||||
* 按智能体代码执行的位置：
|
||||
- **静态**智能体在专用网络节点上工作
|
||||
- **移动**智能体可以在网络节点之间移动其代码
|
||||
* 按其行为：
|
||||
- **被动智能体**没有具体目标。这类智能体可以对外部刺激做出反应，但不会主动采取行动。
|
||||
- **主动智能体**有一些目标并努力实现这些目标
|
||||
- **认知智能体**涉及复杂的规划和推理
|
||||
|
||||
如今，多智能体系统被应用于许多领域：
|
||||
|
||||
* 在游戏中，许多非玩家角色使用某种人工智能，可以被视为智能体
|
||||
* 在视频制作中，渲染涉及人群的复杂3D场景通常使用多智能体模拟
|
||||
* 在系统建模中，多智能体方法用于模拟复杂模型的行为。例如，多智能体方法已成功用于预测COVID-19疾病在全球的传播。类似的方法可以用于模拟城市交通，并观察其对交通规则变化的反应。
|
||||
* 在复杂的自动化系统中，每个设备可以作为一个独立的智能体，这使得整个系统不那么单一且更具鲁棒性。
|
||||
|
||||
我们不会深入探讨多智能体系统，但会考虑一个**多智能体建模**的示例。
|
||||
|
||||
## NetLogo
|
||||
|
||||
[NetLogo](https://ccl.northwestern.edu/netlogo/)是一个基于修改版[Logo](https://en.wikipedia.org/wiki/Logo_(programming_language))编程语言的多智能体建模环境。这种语言是为教授编程概念给孩子而开发的，它允许你控制一个名为**乌龟**的智能体，该智能体可以移动并留下痕迹。这使得创建复杂的几何图形成为可能，这是一种非常直观的方式来理解智能体的行为。
|
||||
|
||||
在NetLogo中，我们可以使用`create-turtles`命令创建许多乌龟。然后我们可以命令所有乌龟执行一些动作（例如下面的例子中向前移动10点）：
|
||||
|
||||
```
|
||||
create-turtles 10
|
||||
ask turtles [
|
||||
forward 10
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
当然，如果所有乌龟都做同样的事情就没什么意思了，所以我们可以`ask`某些乌龟群体，例如那些在某个点附近的乌龟。我们还可以使用`breed [cats cat]`命令创建不同*种类*的乌龟。这里`cat`是种类的名称，我们需要同时指定单数和复数形式，因为不同的命令使用不同的形式以提高清晰度。
|
||||
|
||||
> ✅ 我们不会深入学习NetLogo语言本身——如果你感兴趣，可以访问优秀的[初学者互动NetLogo词典](https://ccl.northwestern.edu/netlogo/bind/)资源。
|
||||
|
||||
你可以[下载](https://ccl.northwestern.edu/netlogo/download.shtml)并安装NetLogo来尝试。
|
||||
|
||||
### 模型库
|
||||
|
||||
NetLogo的一个优点是它包含一个可供尝试的工作模型库。进入**File → Models Library**，你会发现许多类别的模型可供选择。
|
||||
|
||||
<img alt="NetLogo模型库" src="../../../../../translated_images/zh-CN/NetLogo-ModelLib.efe023afb4763c05.webp" width="60%"/>
|
||||
|
||||
> Dmitry Soshnikov提供的模型库截图
|
||||
|
||||
你可以打开其中一个模型，例如**Biology → Flocking**。
|
||||
|
||||
### 主要原则
|
||||
|
||||
打开模型后，你会进入NetLogo的主界面。以下是一个描述狼和羊种群的示例模型，考虑到有限资源（草地）。
|
||||
|
||||
|
||||
|
||||
> Dmitry Soshnikov提供的截图
|
||||
|
||||
在这个界面上，你可以看到：
|
||||
|
||||
* **界面**部分包含：
|
||||
- 智能体生活的主场地
|
||||
- 不同的控件：按钮、滑块等
|
||||
- 用于显示模拟参数的图表
|
||||
* **代码**标签包含编辑器，你可以在其中编写NetLogo程序
|
||||
|
||||
在大多数情况下，界面会有一个**Setup**按钮，用于初始化模拟状态，以及一个**Go**按钮，用于启动执行。这些按钮由代码中的相应处理程序处理，代码如下所示：
|
||||
|
||||
```
|
||||
to go [
|
||||
...
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
NetLogo的世界由以下对象组成：
|
||||
|
||||
* **智能体**（乌龟），可以在场地上移动并执行某些操作。你可以使用`ask turtles [...]`语法命令智能体，括号中的代码由所有智能体以*乌龟模式*执行。
|
||||
* **补丁**是场地上的方块区域，智能体生活在其上。你可以引用同一补丁上的所有智能体，或者更改补丁的颜色和其他属性。你也可以`ask patches`执行某些操作。
|
||||
* **观察者**是一个独特的智能体，控制整个世界。所有按钮处理程序都以*观察者模式*执行。
|
||||
|
||||
> ✅ 多智能体环境的美妙之处在于，以乌龟模式或补丁模式运行的代码会由所有智能体同时并行执行。因此，通过编写少量代码并编程单个智能体的行为，你可以创建整个模拟系统的复杂行为。
|
||||
|
||||
### 群体行为
|
||||
|
||||
作为多智能体行为的一个示例，我们来看看**[群体行为](https://en.wikipedia.org/wiki/Flocking_(behavior))**。群体行为是一种复杂模式，非常类似于鸟群飞行的方式。观察它们飞行时，你可能会认为它们遵循某种集体算法，或者它们拥有某种形式的*集体智能*。然而，这种复杂行为的产生仅仅是因为每个单独的智能体（在这种情况下是*鸟*）只观察短距离内的其他智能体，并遵循三个简单规则：
|
||||
|
||||
* **对齐** - 朝邻近智能体的平均方向转向
|
||||
* **凝聚** - 尝试朝邻居的平均位置转向（*远距离吸引*）
|
||||
* **分离** - 当与其他鸟靠得太近时，尝试远离（*短距离排斥*）
|
||||
|
||||
你可以运行群体行为示例并观察其行为。你还可以调整参数，例如*分离程度*或*视距*，定义每只鸟能看到的范围。注意，如果你将视距减少到0，所有鸟都会变得“盲目”，群体行为会停止。如果你将分离减少到0，所有鸟会聚集成一条直线。
|
||||
|
||||
> ✅ 切换到**代码**标签，查看群体行为的三个规则（对齐、凝聚和分离）如何在代码中实现。注意我们如何仅引用视野范围内的智能体。
|
||||
|
||||
### 其他可查看的模型
|
||||
|
||||
还有一些有趣的模型可以供你实验：
|
||||
|
||||
* **Art → Fireworks**展示了烟花如何被视为单个火流的集体行为
|
||||
* **Social Science → Traffic Basic**和**Social Science → Traffic Grid**展示了城市交通在1D和2D网格中有或没有交通灯的模型。模拟中的每辆车遵循以下规则：
|
||||
- 如果前方空间为空——加速（直到某个最大速度）
|
||||
- 如果看到前方有障碍物——刹车（你可以调整司机的视距）
|
||||
* **Social Science → Party**展示了人们在鸡尾酒会上如何聚集。你可以找到导致群体幸福度最快提升的参数组合。
|
||||
|
||||
从这些示例中可以看出，多智能体模拟可以成为理解由遵循相同或类似逻辑的个体组成的复杂系统行为的一种有用方式。它还可以用于控制虚拟智能体，例如计算机游戏中的[NPC](https://en.wikipedia.org/wiki/NPC)，或3D动画世界中的智能体。
|
||||
|
||||
## 推理型智能体
|
||||
|
||||
上述智能体非常简单，通过某种算法对环境变化做出反应。因此它们是**反应型智能体**。然而，有时智能体可以进行推理和规划其行动，这种情况下它们被称为**推理型智能体**。
|
||||
|
||||
一个典型的例子是个人智能体，它接收到人类的指令去预订一个度假行程。假设互联网上有许多智能体可以帮助它。它需要联系其他智能体以查看哪些航班可用，不同日期的酒店价格是多少，并尝试协商最佳价格。当度假计划完成并得到主人确认后，它可以继续预订。
|
||||
|
||||
为了实现这一点，智能体需要**通信**。为了成功通信，它们需要：
|
||||
|
||||
* 一些**标准语言来交换知识**，例如[知识交换格式](https://en.wikipedia.org/wiki/Knowledge_Interchange_Format)（KIF）和[知识查询与操作语言](https://en.wikipedia.org/wiki/Knowledge_Query_and_Manipulation_Language)（KQML）。这些语言基于[言语行为理论](https://en.wikipedia.org/wiki/Speech_act)设计。
|
||||
* 这些语言还应包括一些**协商协议**，基于不同的**拍卖类型**。
|
||||
* 一个**通用本体**，以便它们引用相同的概念并了解其语义
|
||||
* 一种**发现**不同智能体功能的方法，也基于某种本体
|
||||
|
||||
推理型智能体比反应型智能体复杂得多，因为它们不仅对环境变化做出反应，还需要能够*主动*采取行动。一种提出的推理型智能体架构是所谓的信念-愿望-意图（BDI）智能体：
|
||||
|
||||
* **信念**构成了智能体对环境的知识集合。它可以被结构化为知识库或智能体可以应用于环境中特定情况的一组规则。
|
||||
* **愿望**定义了智能体想要做什么，即它的目标。例如，上述个人助理智能体的目标是预订旅行，而酒店智能体的目标是最大化利润。
|
||||
* **意图**是智能体为实现其目标而计划的具体行动。行动通常会改变环境并导致与其他智能体的通信。
|
||||
|
||||
有一些平台可用于构建多智能体系统，例如[JADE](https://jade.tilab.com/)。[这篇论文](https://arxiv.org/ftp/arxiv/papers/2007/2007.08961.pdf)包含了多智能体平台的综述，以及多智能体系统的简史及其不同的使用场景。
|
||||
|
||||
## 结论
|
||||
|
||||
多智能体系统可以采取非常不同的形式，并应用于许多不同的领域。
|
||||
它们通常专注于单个智能体的简单行为，并通过**协同效应**实现整个系统的更复杂行为。
|
||||
|
||||
## 🚀 挑战
|
||||
|
||||
将本课内容应用于现实世界，尝试构思一个可以解决问题的多智能体系统。例如，多智能体系统需要做些什么来优化校车路线？它如何在面包店中工作？
|
||||
|
||||
## [课后测验](https://ff-quizzes.netlify.app/en/ai/quiz/46)
|
||||
|
||||
## 复习与自学
|
||||
|
||||
复习这种系统在工业中的应用。选择一个领域，例如制造业或视频游戏行业，探索多智能体系统如何用于解决独特问题。
|
||||
|
||||
## [NetLogo作业](assignment.md)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
# NetLogo 作业
|
||||
|
||||
从 NetLogo 的库中选择一个模型，并使用它尽可能接近地模拟一个现实生活中的情境。一个很好的例子是调整“替代可视化”文件夹中的病毒模型，展示它如何用于模拟 COVID-19 的传播。你能否构建一个模拟现实生活中病毒传播的模型？
|
||||
|
||||
通过保存一个副本并制作一个视频演示来展示你的工作，解释该模型如何与现实世界的情境相关联。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。因使用本翻译而引起的任何误解或误读，我们概不负责。
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
# 伦理与负责任的人工智能
|
||||
|
||||
你已经快完成这门课程了，希望到现在为止，你已经清楚地了解到人工智能是基于一系列正式的数学方法，这些方法使我们能够发现数据中的关系，并训练模型以复制人类行为的某些方面。在当下，我们认为人工智能是一种非常强大的工具，可以从数据中提取模式，并将这些模式应用于解决新的问题。
|
||||
|
||||
## [课前测验](https://white-water-09ec41f0f.azurestaticapps.net/quiz/5/)
|
||||
|
||||
然而，在科幻作品中，我们经常看到人工智能对人类构成威胁的故事。这些故事通常围绕某种人工智能叛变展开，即人工智能决定与人类对抗。这暗示人工智能具有某种情感或能够做出开发者未曾预料的决定。
|
||||
|
||||
我们在这门课程中学习的人工智能仅仅是大型矩阵运算。它是一种非常强大的工具，可以帮助我们解决问题，但和其他任何强大的工具一样——它既可以被用于好的目的，也可以被用于坏的目的。重要的是，它可能会被*滥用*。
|
||||
|
||||
## 负责任人工智能的原则
|
||||
|
||||
为了避免人工智能的意外或故意滥用，微软提出了重要的[负责任人工智能原则](https://www.microsoft.com/ai/responsible-ai?WT.mc_id=academic-77998-cacaste)。以下概念是这些原则的基础：
|
||||
|
||||
* **公平性**与*模型偏差*这一重要问题相关，这种偏差可能是由于使用了有偏的数据进行训练。例如，当我们试图预测一个人获得软件开发职位的概率时，模型可能会更倾向于男性——仅仅因为训练数据集可能偏向男性群体。我们需要仔细平衡训练数据并调查模型以避免偏差，并确保模型考虑到更相关的特征。
|
||||
* **可靠性与安全性**。人工智能模型本质上可能会犯错。神经网络返回的是概率，我们需要在做决策时考虑到这一点。每个模型都有一定的精确度和召回率，我们需要理解这些指标，以防止错误建议可能造成的伤害。
|
||||
* **隐私与安全性**具有一些人工智能特定的影响。例如，当我们使用某些数据训练模型时，这些数据会以某种方式“整合”到模型中。一方面，这提高了安全性和隐私性，另一方面，我们需要记住模型是基于哪些数据训练的。
|
||||
* **包容性**意味着我们不是在构建人工智能来取代人类，而是为了增强人类的能力，使我们的工作更具创造性。这也与公平性相关，因为在处理代表性不足的群体时，我们收集的大多数数据集可能会存在偏差，我们需要确保这些群体被包括在内并被人工智能正确处理。
|
||||
* **透明性**。这包括确保我们始终明确人工智能的使用。此外，在可能的情况下，我们希望使用*可解释*的人工智能系统。
|
||||
* **问责性**。当人工智能模型做出某些决定时，责任归属并不总是清晰的。我们需要确保理解人工智能决策的责任归属。在大多数情况下，我们希望将人类纳入重要决策的环节，以便实际的人类承担责任。
|
||||
|
||||
## 负责任人工智能的工具
|
||||
|
||||
微软开发了[负责任人工智能工具箱](https://github.com/microsoft/responsible-ai-toolbox)，其中包含一系列工具：
|
||||
|
||||
* 可解释性仪表板 (InterpretML)
|
||||
* 公平性仪表板 (FairLearn)
|
||||
* 错误分析仪表板
|
||||
* 负责任人工智能仪表板，包括：
|
||||
|
||||
- EconML - 用于因果分析的工具，专注于假设性问题
|
||||
- DiCE - 用于反事实分析的工具，允许你查看需要更改哪些特征以影响模型的决策
|
||||
|
||||
关于人工智能伦理的更多信息，请访问机器学习课程中的[这一课](https://github.com/microsoft/ML-For-Beginners/tree/main/1-Introduction/3-fairness?WT.mc_id=academic-77998-cacaste)，其中包括相关作业。
|
||||
|
||||
## 复习与自学
|
||||
|
||||
通过这个[学习路径](https://docs.microsoft.com/learn/modules/responsible-ai-principles/?WT.mc_id=academic-77998-cacaste)了解更多关于负责任人工智能的内容。
|
||||
|
||||
## [课后测验](https://white-water-09ec41f0f.azurestaticapps.net/quiz/6/)
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
# 概述
|
||||
|
||||
|
||||
|
||||
> 手绘笔记由 [Tomomi Imura](https://twitter.com/girlie_mac) 提供
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,79 @@
|
|||
# 多模态网络
|
||||
|
||||
在Transformer模型成功解决NLP任务后，相同或类似的架构也被应用于计算机视觉任务。越来越多的人开始关注构建能够*结合*视觉和自然语言能力的模型。其中一个尝试是由OpenAI完成的，称为CLIP和DALL.E。
|
||||
|
||||
## 对比图像预训练（CLIP）
|
||||
|
||||
CLIP的核心思想是能够比较文本提示与图像，并确定图像与提示的匹配程度。
|
||||
|
||||
|
||||
|
||||
> *图片来源于[这篇博客](https://openai.com/blog/clip/)*
|
||||
|
||||
该模型通过从互联网获取的图像及其标题进行训练。对于每个批次，我们获取N对（图像，文本），并将它们转换为一些向量表示I，..., I / T，..., T。这些表示随后进行匹配。损失函数的定义是最大化对应一对（例如I和T）的向量之间的余弦相似度，同时最小化所有其他对之间的余弦相似度。这就是为什么这种方法被称为**对比学习**。
|
||||
|
||||
CLIP模型/库可以从[OpenAI GitHub](https://github.com/openai/CLIP)获取。该方法在[这篇博客](https://openai.com/blog/clip/)中有所描述，并在[这篇论文](https://arxiv.org/pdf/2103.00020.pdf)中有更详细的说明。
|
||||
|
||||
一旦该模型被预训练，我们可以给它一批图像和一批文本提示，它将返回一个概率张量。CLIP可以用于以下任务：
|
||||
|
||||
**图像分类**
|
||||
|
||||
假设我们需要将图像分类为猫、狗和人类。在这种情况下，我们可以给模型一个图像，以及一系列文本提示：“*一张猫的图片*”、“*一张狗的图片*”、“*一张人类的图片*”。在结果的3个概率向量中，我们只需选择值最高的索引。
|
||||
|
||||
|
||||
|
||||
> *图片来源于[这篇博客](https://openai.com/blog/clip/)*
|
||||
|
||||
**基于文本的图像搜索**
|
||||
|
||||
我们也可以反过来操作。如果我们有一组图像，我们可以将这组图像传递给模型，并提供一个文本提示——这将返回与给定提示最相似的图像。
|
||||
|
||||
## ✍️ 示例：[使用CLIP进行图像分类和图像搜索](../../../../../lessons/X-Extras/X1-MultiModal/Clip.ipynb)
|
||||
|
||||
打开[Clip.ipynb](../../../../../lessons/X-Extras/X1-MultiModal/Clip.ipynb)笔记本，查看CLIP的实际应用。
|
||||
|
||||
## 使用VQGAN+CLIP进行图像生成
|
||||
|
||||
CLIP还可以用于从文本提示生成**图像**。为了实现这一点，我们需要一个**生成器模型**，能够根据某些向量输入生成图像。其中一个这样的模型称为[VQGAN](https://compvis.github.io/taming-transformers/)（向量量化GAN）。
|
||||
|
||||
VQGAN与普通[GAN](../../4-ComputerVision/10-GANs/README.md)的主要区别在于以下几点：
|
||||
* 使用自回归Transformer架构生成一系列上下文丰富的视觉部分，这些部分组成图像。这些视觉部分由[卷积神经网络（CNN）](../../4-ComputerVision/07-ConvNets/README.md)学习。
|
||||
* 使用子图像判别器来检测图像的部分是“真实”还是“伪造”（与传统GAN的“全或无”方法不同）。
|
||||
|
||||
可以在[Taming Transformers](https://compvis.github.io/taming-transformers/)网站上了解更多关于VQGAN的信息。
|
||||
|
||||
VQGAN与传统GAN的一个重要区别在于，后者可以从任何输入向量生成一个不错的图像，而VQGAN可能生成一个不连贯的图像。因此，我们需要进一步引导图像创建过程，这可以通过CLIP来实现。
|
||||
|
||||
|
||||
|
||||
为了生成与文本提示相对应的图像，我们从一些随机编码向量开始，将其传递给VQGAN以生成图像。然后使用CLIP生成一个损失函数，显示图像与文本提示的匹配程度。目标是通过反向传播调整输入向量参数以最小化该损失。
|
||||
|
||||
一个实现VQGAN+CLIP的优秀库是[Pixray](http://github.com/pixray/pixray)。
|
||||
|
||||
 |  | 
|
||||
----|----|----
|
||||
从提示*一张年轻男性文学教师拿着书的水彩特写肖像*生成的图片 | 从提示*一张年轻女性计算机科学教师拿着电脑的油画特写肖像*生成的图片 | 从提示*一张老年男性数学教师站在黑板前的油画特写肖像*生成的图片
|
||||
|
||||
> 图片来自**人工教师**系列，由[Dmitry Soshnikov](http://soshnikov.com)创作
|
||||
|
||||
## DALL-E
|
||||
### [DALL-E 1](https://openai.com/research/dall-e)
|
||||
DALL-E是一个基于GPT-3的版本，能够根据提示生成图像。它使用了120亿参数进行训练。
|
||||
|
||||
与CLIP不同，DALL-E将文本和图像作为一个单一的令牌流输入。因此，可以根据多个提示生成图像。
|
||||
|
||||
### [DALL-E 2](https://openai.com/dall-e-2)
|
||||
DALL-E 1和2的主要区别在于，后者能够生成更真实的图像和艺术作品。
|
||||
|
||||
以下是使用DALL-E生成图像的示例：
|
||||
 |  | 
|
||||
----|----|----
|
||||
从提示*一张年轻男性文学教师拿着书的水彩特写肖像*生成的图片 | 从提示*一张年轻女性计算机科学教师拿着电脑的油画特写肖像*生成的图片 | 从提示*一张老年男性数学教师站在黑板前的油画特写肖像*生成的图片
|
||||
|
||||
## 参考文献
|
||||
|
||||
* VQGAN论文：[Taming Transformers for High-Resolution Image Synthesis](https://compvis.github.io/taming-transformers/paper/paper.pdf)
|
||||
* CLIP论文：[Learning Transferable Visual Models From Natural Language Supervision](https://arxiv.org/pdf/2103.00020.pdf)
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,188 @@
|
|||
归属-相同方式共享 4.0 国际
|
||||
|
||||
=======================================================================
|
||||
|
||||
创作共用公司（“创作共用”）不是律师事务所，也不提供法律服务或法律建议。分发创作共用公共许可不会建立律师-客户或其他关系。创作共用以“按原样”方式提供其许可及相关信息。创作共用对其许可、根据其条款和条件许可的任何材料或任何相关信息不作任何保证。创作共用在法律允许的最大范围内对因使用其许可而导致的损害不承担任何责任。
|
||||
|
||||
使用创作共用公共许可
|
||||
|
||||
创作共用公共许可提供了一套标准条款和条件，创作者和其他权利持有人可以使用这些条款和条件来分享受版权保护的原创作品以及公共许可中规定的某些其他权利。以下注意事项仅供参考，并不详尽，也不构成我们许可的一部分。
|
||||
|
||||
对许可人的注意事项：我们的公共许可旨在供那些有权向公众授权使用受版权和某些其他权利限制的材料的人使用。我们的许可是不可撤销的。许可人在应用许可之前应阅读并理解所选许可的条款和条件。许可人还应在应用我们的许可之前确保获得所有必要的权利，以便公众能够按预期重新使用材料。许可人应清楚标明任何不受许可约束的材料，包括其他创作共用许可的材料，或根据版权的例外或限制使用的材料。更多关于许可人的注意事项：
|
||||
wiki.creativecommons.org/Considerations_for_licensors
|
||||
|
||||
对公众的注意事项：通过使用我们的公共许可，许可人授予公众在规定的条款和条件下使用许可材料的权限。如果因任何原因不需要许可人的授权，例如因适用的版权例外或限制，则该使用不受许可约束。我们的许可仅授予许可人有权授予的版权和某些其他权利下的权限。对许可材料的使用可能仍因其他原因受到限制，包括其他人对材料拥有版权或其他权利。许可人可能会提出特别要求，例如要求标明或描述所有更改。虽然我们的许可不要求这样做，但建议在合理的情况下尊重这些要求。更多关于公众的注意事项：
|
||||
wiki.creativecommons.org/Considerations_for_licensees
|
||||
|
||||
=======================================================================
|
||||
|
||||
创作共用归属-相同方式共享 4.0 国际公共许可
|
||||
|
||||
通过行使许可权利（定义如下），您接受并同意受创作共用归属-相同方式共享 4.0 国际公共许可（“公共许可”）条款和条件的约束。如果此公共许可可被解释为合同，则您因接受这些条款和条件而被授予许可权利，许可人因根据这些条款和条件提供许可材料而获得的利益授予您这些权利。
|
||||
|
||||
第 1 节 -- 定义。
|
||||
|
||||
a. 改编材料指受版权及类似权利约束的材料，这些材料基于许可材料进行翻译、修改、编排、转化或以其他方式变更，需获得许可人持有的版权及类似权利的许可。对于本公共许可而言，当许可材料是音乐作品、表演或录音时，改编材料总是在许可材料与动态影像同步时产生。
|
||||
|
||||
b. 改编者许可指您根据本公共许可的条款和条件对您在改编材料中的贡献应用的版权及类似权利许可。
|
||||
|
||||
c. BY-SA 兼容许可指创作共用批准的、与本公共许可基本等同的许可，列于 creativecommons.org/compatiblelicenses。
|
||||
|
||||
d. 版权及类似权利指与版权密切相关的版权和/或类似权利，包括但不限于表演、广播、录音和特殊数据库权利，无论这些权利如何被标记或分类。对于本公共许可而言，第 2(b)(1)-(2) 中规定的权利不属于版权及类似权利。
|
||||
|
||||
e. 有效技术措施指在没有适当授权的情况下，根据 1996 年 12 月 20 日通过的《世界知识产权组织版权条约》第 11 条及/或类似国际协议的义务，无法规避的措施。
|
||||
|
||||
f. 例外和限制指适用于您使用许可材料的版权及类似权利的任何例外或限制，例如合理使用、合理交易等。
|
||||
|
||||
g. 许可元素指创作共用公共许可名称中列出的许可属性。本公共许可的许可元素为归属和相同方式共享。
|
||||
|
||||
h. 许可材料指许可人应用本公共许可的艺术或文学作品、数据库或其他材料。
|
||||
|
||||
i. 许可权利指根据本公共许可的条款和条件授予您的权利，这些权利仅限于适用于您使用许可材料的所有版权及类似权利，并且许可人有权许可这些权利。
|
||||
|
||||
j. 许可人指根据本公共许可授予权利的个人或实体。
|
||||
|
||||
k. 分享指通过任何需要许可权利的方式或过程向公众提供材料，例如复制、公开展示、公开表演、分发、传播、通信或进口，以及以公众可以在其选择的时间和地点访问材料的方式向公众提供材料。
|
||||
|
||||
l. 特殊数据库权利指除版权外的权利，这些权利源自 1996 年 3 月 11 日欧洲议会和理事会通过的《数据库法律保护指令》（96/9/EC），以及其修订或继承版本，以及世界范围内其他基本等同的权利。
|
||||
|
||||
m. 您指根据本公共许可行使许可权利的个人或实体。“您的”具有相应的含义。
|
||||
|
||||
第 2 节 -- 范围。
|
||||
|
||||
a. 许可授予。
|
||||
|
||||
1. 根据本公共许可的条款和条件，许可人特此授予您全球范围内的、免版税的、不可转授权的、非独占的、不可撤销的许可，以行使许可权利：
|
||||
|
||||
a. 复制和分享许可材料，无论是全部还是部分；以及
|
||||
|
||||
b. 生产、复制和分享改编材料。
|
||||
|
||||
2. 例外和限制。为避免疑义，如果例外和限制适用于您的使用，则本公共许可不适用，您无需遵守其条款和条件。
|
||||
|
||||
3. 期限。本公共许可的期限在第 6(a) 节中规定。
|
||||
|
||||
4. 媒体和格式；允许技术修改。许可人授权您在现有或未来创建的所有媒体和格式中行使许可权利，并进行必要的技术修改。许可人放弃和/或同意不主张任何权利或权限，以禁止您进行必要的技术修改以行使许可权利，包括必要的技术修改以规避有效技术措施。对于本公共许可，仅进行本第 2(a)(4) 节授权的修改从未产生改编材料。
|
||||
|
||||
5. 下游接收者。
|
||||
|
||||
a. 许可人的要约 -- 许可材料。每个许可材料的接收者自动收到许可人的要约，以根据本公共许可的条款和条件行使许可权利。
|
||||
|
||||
b. 许可人的额外要约 -- 改编材料。每个从您处接收改编材料的接收者自动收到许可人的要约，以根据您应用的改编者许可的条件行使改编材料中的许可权利。
|
||||
|
||||
c. 无下游限制。您不得对许可材料施加任何额外或不同的条款和条件，也不得应用任何有效技术措施，如果这样做会限制任何接收者行使许可权利。
|
||||
|
||||
6. 无认可。本公共许可中的任何内容均不构成或可解释为允许您主张或暗示您或您的使用许可材料与许可人或其他指定接收归属的人有联系、获得认可或授予官方地位。
|
||||
|
||||
b. 其他权利。
|
||||
|
||||
1. 道德权利，例如完整性权利，不在本公共许可范围内，也不包括宣传权、隐私权和/或其他类似的个性权利；然而，在可能的范围内，许可人放弃和/或同意不主张许可人持有的任何此类权利，以允许您在有限范围内行使许可权利，但除此之外不适用。
|
||||
|
||||
2. 专利和商标权不在本公共许可范围内。
|
||||
|
||||
3. 在可能的范围内，许可人放弃任何直接或通过收集协会根据任何自愿或可放弃的法定或强制许可计划向您收取版税的权利。在所有其他情况下，许可人明确保留收取此类版税的权利。
|
||||
|
||||
第 3 节 -- 许可条件。
|
||||
|
||||
您行使许可权利明确以以下条件为前提。
|
||||
|
||||
a. 归属。
|
||||
|
||||
1. 如果您分享许可材料（包括修改后的形式），您必须：
|
||||
|
||||
a. 保留以下内容（如果许可人随许可材料提供）：
|
||||
|
||||
i. 许可材料创作者及任何其他指定接收归属者的身份信息，以许可人要求的任何合理方式（包括使用化名，如果指定）；
|
||||
|
||||
ii. 版权声明；
|
||||
|
||||
iii. 提及本公共许可的声明；
|
||||
|
||||
iv. 提及免责声明的声明；
|
||||
|
||||
v. 在合理可行的范围内，许可材料的 URI 或超链接；
|
||||
|
||||
b. 表明您是否修改了许可材料，并保留任何先前修改的标识；
|
||||
|
||||
c. 表明许可材料是根据本公共许可授权的，并包括本公共许可的文本或 URI 或超链接。
|
||||
|
||||
2. 您可以根据您分享许可材料的媒介、方式和上下文，以任何合理的方式满足第 3(a)(1) 节中的条件。例如，可以通过提供包含所需信息的资源的 URI 或超链接来满足条件。
|
||||
|
||||
3. 如果许可人要求，您必须在合理可行的范围内删除第 3(a)(1)(A) 节中要求的任何信息。
|
||||
|
||||
b. 相同方式共享。
|
||||
|
||||
除第 3(a) 节中的条件外，如果您分享您制作的改编材料，还需满足以下条件。
|
||||
|
||||
1. 您应用的改编者许可必须是具有相同许可元素的创作共用许可（本版本或更高版本），或 BY-SA 兼容许可。
|
||||
|
||||
2. 您必须包括您应用的改编者许可的文本或 URI 或超链接。您可以根据您分享改编材料的媒介、方式和上下文，以任何合理的方式满足此条件。
|
||||
|
||||
3. 您不得对改编材料施加任何额外或不同的条款和条件，也不得应用任何有效技术措施，如果这样做会限制根据您应用的改编者许可授予的权利。
|
||||
|
||||
第 4 节 -- 特殊数据库权利。
|
||||
|
||||
如果许可权利包括适用于您使用许可材料的特殊数据库权利：
|
||||
|
||||
a. 为避免疑义，第 2(a)(1) 节授予您提取、重用、复制和分享数据库内容的全部或实质部分的权利；
|
||||
|
||||
b. 如果您将数据库内容的全部或实质部分包含在您拥有特殊数据库权利的数据库中：
|
||||
权利，然后您对数据库拥有的特殊数据库权利（但不包括其单独内容）属于改编材料，
|
||||
|
||||
包括用于第 3(b) 节的目的；并且
|
||||
c. 如果您共享数据库的全部或大部分内容，您必须遵守第 3(a) 节中的条件。
|
||||
|
||||
为避免疑义，本第 4 节是对您在本公共许可下的义务的补充，而不是替代，当许可权利包括其他版权和类似权利时。
|
||||
|
||||
|
||||
第 5 节 —— 免责声明和责任限制。
|
||||
|
||||
a. 除非许可方另有单独承诺，在可能的范围内，许可方按“现状”和“可用”状态提供许可材料，并且不对许可材料作出任何形式的陈述或保证，无论是明示、暗示、法定或其他形式的。这包括但不限于所有权保证、适销性、特定用途适用性、非侵权性、无潜在或其他缺陷、准确性或是否存在错误（无论是否已知或可发现）。如果法律不允许完全或部分免责声明，则此免责声明可能不适用于您。
|
||||
|
||||
b. 在可能的范围内，无论基于何种法律理论（包括但不限于过失）或其他原因，许可方在任何情况下均不对因本公共许可或使用许可材料而导致的任何直接、特殊、间接、附带、后果性、惩罚性、示范性或其他损失、成本、费用或损害承担责任，即使许可方已被告知可能发生此类损失、成本、费用或损害。如果法律不允许完全或部分责任限制，则此限制可能不适用于您。
|
||||
|
||||
c. 上述免责声明和责任限制应以尽可能接近绝对免责声明和放弃所有责任的方式进行解释。
|
||||
|
||||
|
||||
第 6 节 —— 期限和终止。
|
||||
|
||||
a. 本公共许可适用于此处许可的版权和类似权利的期限。然而，如果您未能遵守本公共许可，则您在本公共许可下的权利将自动终止。
|
||||
|
||||
b. 如果根据第 6(a) 节您的使用许可材料的权利已终止，则该权利可恢复：
|
||||
|
||||
1. 如果在您发现违规行为后 30 天内纠正了违规行为，则自纠正之日起自动恢复；或
|
||||
2. 经许可方明确恢复。
|
||||
|
||||
为避免疑义，本第 6(b) 节不影响许可方因您违反本公共许可而寻求补救的任何权利。
|
||||
|
||||
c. 为避免疑义，许可方也可以根据单独的条款或条件提供许可材料，或随时停止分发许可材料；然而，这不会终止本公共许可。
|
||||
|
||||
d. 第 1、5、6、7 和 8 节在本公共许可终止后仍然有效。
|
||||
|
||||
|
||||
第 7 节 —— 其他条款和条件。
|
||||
|
||||
a. 除非许可方明确同意，否则许可方不受您传达的任何附加或不同条款或条件的约束。
|
||||
|
||||
b. 关于许可材料的任何安排、理解或协议，如果未在此处说明，则与本公共许可的条款和条件分开且独立。
|
||||
|
||||
|
||||
第 8 节 —— 解释。
|
||||
|
||||
a. 为避免疑义，本公共许可不会且不应被解释为减少、限制、约束或对任何在未获得本公共许可许可的情况下可以合法使用许可材料的行为施加条件。
|
||||
|
||||
b. 在可能的范围内，如果本公共许可的任何条款被视为不可执行，则应自动以最低限度的方式进行修改以使其可执行。如果该条款无法修改，则应从本公共许可中删除，而不影响其余条款和条件的可执行性。
|
||||
|
||||
c. 除非许可方明确同意，否则本公共许可的任何条款或条件均不得被放弃，也不得因未遵守而被视为同意。
|
||||
|
||||
d. 本公共许可中的任何内容均不构成或可被解释为对适用于许可方或您的任何特权和豁免的限制或放弃，包括来自任何司法管辖区或权威机构的法律程序的豁免。
|
||||
|
||||
|
||||
=======================================================================
|
||||
|
||||
Creative Commons 不是其公共许可的当事方。然而，Creative Commons 可以选择将其公共许可应用于其发布的材料，在这些情况下将被视为“许可方”。Creative Commons 公共许可的文本已通过 CC0 公共领域奉献声明献给公共领域。除非用于表明材料是根据 Creative Commons 公共许可共享的有限目的，或根据 Creative Commons 在 creativecommons.org/policies 上发布的政策另行允许，Creative Commons 不授权使用“Creative Commons”商标或任何其他 Creative Commons 的商标或标志，未经其事先书面同意，包括但不限于与对其任何公共许可的任何未经授权的修改或与许可材料使用相关的任何其他安排、理解或协议相关的情况。为避免疑义，本段不构成公共许可的一部分。
|
||||
|
||||
您可以通过 creativecommons.org 联系 Creative Commons。
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。我们对于因使用本翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
所有课程的手绘笔记可以在这里下载。
|
||||
|
||||
🎨 由以下作者创作：Tomomi Imura（Twitter: [@girlie_mac](https://twitter.com/girlie_mac)，GitHub: [girliemac](https://github.com/girliemac)）
|
||||
|
||||
[](https://creativecommons.org/licenses/by-sa/4.0/)
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原文档的原始语言版本为权威来源。对于关键信息，建议使用专业人工翻译。我们对于因使用本翻译而引起的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,278 @@
|
|||
# AI-For-Beginners 故障排查指南
|
||||
|
||||
本指南旨在帮助您解决使用或贡献 [AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners) 仓库时遇到的常见问题。每个问题都包括背景信息、症状、原因解释以及逐步解决方案。
|
||||
|
||||
---
|
||||
|
||||
## 目录
|
||||
|
||||
- [常见问题](../..)
|
||||
- [安装问题](../..)
|
||||
- [配置问题](../..)
|
||||
- [运行笔记本](../..)
|
||||
- [性能问题](../..)
|
||||
- [教材网站问题](../..)
|
||||
- [贡献问题](../..)
|
||||
- [常见问答](../..)
|
||||
- [获取帮助](../..)
|
||||
|
||||
---
|
||||
|
||||
## 常见问题
|
||||
|
||||
### 1. 仓库无法正确克隆
|
||||
|
||||
**背景：** 克隆操作可以将仓库复制到您的机器上。
|
||||
|
||||
**症状：**
|
||||
- 错误：`fatal: repository not found`
|
||||
- 错误：`Permission denied (publickey)`
|
||||
|
||||
**可能原因：**
|
||||
- 仓库 URL 不正确
|
||||
- 权限不足
|
||||
- SSH 密钥未配置
|
||||
|
||||
**解决方案：**
|
||||
1. **检查仓库 URL。**
|
||||
使用 HTTPS URL：
|
||||
```
|
||||
git clone https://github.com/microsoft/AI-For-Beginners.git
|
||||
```
|
||||
2. **如果 SSH 失败，切换到 HTTPS。**
|
||||
如果出现 `Permission denied (publickey)` 错误，请使用上述 HTTPS 链接代替 SSH。
|
||||
3. **配置 SSH 密钥（可选）。**
|
||||
如果您希望使用 SSH，请参考 [GitHub 的 SSH 指南](https://docs.github.com/en/authentication/connecting-to-github-with-ssh)。
|
||||
|
||||
---
|
||||
|
||||
## 安装问题
|
||||
|
||||
### 2. Python 环境问题
|
||||
|
||||
**背景：** 仓库依赖于 Python 和各种库。
|
||||
|
||||
**症状：**
|
||||
- 错误：`ModuleNotFoundError: No module named '<package>'`
|
||||
- 运行脚本或笔记本时出现导入错误
|
||||
|
||||
**可能原因：**
|
||||
- 依赖未安装
|
||||
- Python 版本错误
|
||||
|
||||
**解决方案：**
|
||||
1. **设置虚拟环境。**
|
||||
```bash
|
||||
python -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
```
|
||||
2. **安装依赖项。**
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
3. **检查 Python 版本。**
|
||||
使用 Python 3.7 或更高版本。
|
||||
```bash
|
||||
python --version
|
||||
```
|
||||
|
||||
### 3. 未安装 Jupyter
|
||||
|
||||
**背景：** 笔记本是核心学习资源。
|
||||
|
||||
**症状：**
|
||||
- 错误：`jupyter: command not found`
|
||||
- 笔记本无法启动
|
||||
|
||||
**可能原因：**
|
||||
- 未安装 Jupyter
|
||||
|
||||
**解决方案：**
|
||||
1. **安装 Jupyter Notebook。**
|
||||
```bash
|
||||
pip install notebook
|
||||
```
|
||||
或者，如果使用 Anaconda：
|
||||
```bash
|
||||
conda install notebook
|
||||
```
|
||||
2. **启动 Jupyter Notebook。**
|
||||
```bash
|
||||
jupyter notebook
|
||||
```
|
||||
|
||||
### 4. 依赖版本冲突
|
||||
|
||||
**背景：** 如果包版本不匹配，项目可能会出现问题。
|
||||
|
||||
**症状：**
|
||||
- 关于版本不兼容的错误或警告
|
||||
|
||||
**可能原因：**
|
||||
- 旧的或冲突的 Python 包
|
||||
|
||||
**解决方案：**
|
||||
1. **在干净的环境中安装。**
|
||||
删除旧的 venv/conda 环境并创建一个新的。
|
||||
2. **使用指定版本。**
|
||||
始终运行：
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
如果失败，请按照 README 手动安装缺失的包。
|
||||
|
||||
---
|
||||
|
||||
## 配置问题
|
||||
|
||||
### 5. 环境变量未设置
|
||||
|
||||
**背景：** 某些模块可能需要密钥、令牌或配置设置。
|
||||
|
||||
**症状：**
|
||||
- 错误：`KeyError` 或关于缺少配置的警告
|
||||
|
||||
**可能原因：**
|
||||
- 必需的环境变量未设置
|
||||
|
||||
**解决方案：**
|
||||
1. **检查 `.env.example` 或类似文件。**
|
||||
2. **创建 `.env` 文件并填写所需值。**
|
||||
3. **设置环境变量后重新加载终端或 IDE。**
|
||||
|
||||
---
|
||||
|
||||
## 运行笔记本
|
||||
|
||||
### 6. 笔记本无法打开或运行
|
||||
|
||||
**背景：** Jupyter 笔记本需要正确的设置。
|
||||
|
||||
**症状：**
|
||||
- 笔记本无法启动
|
||||
- 浏览器未自动打开
|
||||
|
||||
**可能原因：**
|
||||
- 未安装 Jupyter
|
||||
- 浏览器配置问题
|
||||
|
||||
**解决方案：**
|
||||
1. **安装 Jupyter（参见安装问题部分）。**
|
||||
2. **手动打开笔记本。**
|
||||
- 从终端复制 URL（例如 `http://localhost:8888/?token=...`），并将其粘贴到浏览器中。
|
||||
|
||||
### 7. 内核崩溃或冻结
|
||||
|
||||
**背景：** 笔记本内核可能因资源限制或代码错误而崩溃。
|
||||
|
||||
**症状：**
|
||||
- 内核反复崩溃或重启
|
||||
- 内存不足错误
|
||||
|
||||
**可能原因：**
|
||||
- 数据集过大
|
||||
- 不兼容的代码或包
|
||||
|
||||
**解决方案：**
|
||||
1. **重启内核。**
|
||||
使用 Jupyter 中的“重启内核”按钮。
|
||||
2. **检查内存使用情况。**
|
||||
关闭未使用的应用程序。
|
||||
3. **在云平台上运行笔记本。**
|
||||
使用 [Google Colab](https://colab.research.google.com/) 或 [Azure Notebooks](https://notebooks.azure.com/)。
|
||||
|
||||
---
|
||||
|
||||
## 性能问题
|
||||
|
||||
### 8. 笔记本运行缓慢
|
||||
|
||||
**背景：** 某些 AI 任务需要大量内存和 CPU。
|
||||
|
||||
**症状：**
|
||||
- 执行速度慢
|
||||
- 笔记本电脑风扇声音很大
|
||||
|
||||
**可能原因：**
|
||||
- 数据集或模型过大
|
||||
- 系统资源有限
|
||||
|
||||
**解决方案：**
|
||||
1. **使用云平台。**
|
||||
- 将笔记本上传到 Colab 或 Azure Notebooks。
|
||||
2. **减少数据集大小。**
|
||||
- 使用样本数据进行练习。
|
||||
3. **关闭不必要的程序。**
|
||||
- 释放系统内存。
|
||||
|
||||
---
|
||||
|
||||
## 教材网站问题
|
||||
|
||||
### 9. 章节无法加载
|
||||
|
||||
**背景：** 在线教材显示课程和章节内容。
|
||||
|
||||
**症状：**
|
||||
- 某章节（例如 Transformers/BERT）缺失或无法打开
|
||||
|
||||
**已知问题：**
|
||||
- [问题 #303](https://github.com/microsoft/AI-For-Beginners/issues/303)：“第 18 章 Transformers. BERT 无法在教材网站上打开。” 原因是文件名错误（`READMEtransformers.md` 而不是 `README.md`）。
|
||||
|
||||
**解决方案：**
|
||||
1. **检查文件重命名错误。**
|
||||
如果您是贡献者，请确保章节文件命名为 `README.md`。
|
||||
2. **报告缺失文件。**
|
||||
在 GitHub 上提交问题，注明章节名称和错误详情。
|
||||
|
||||
---
|
||||
|
||||
## 贡献问题
|
||||
|
||||
### 10. PR 未被接受或构建失败
|
||||
|
||||
**背景：** 贡献必须通过测试并遵循指南。
|
||||
|
||||
**症状：**
|
||||
- 拉取请求被拒绝
|
||||
- CI/CD 管道错误
|
||||
|
||||
**可能原因：**
|
||||
- 测试失败
|
||||
- 未遵循编码标准
|
||||
|
||||
**解决方案：**
|
||||
1. **阅读贡献指南。**
|
||||
- 遵循仓库的 [CONTRIBUTING.md](https://github.com/microsoft/AI-For-Beginners/blob/main/CONTRIBUTING.md)。
|
||||
2. **在推送之前本地运行测试。**
|
||||
3. **检查代码格式或 linting 规则。**
|
||||
|
||||
---
|
||||
|
||||
## 常见问答
|
||||
|
||||
### 哪里可以找到特定模块的帮助？
|
||||
- 每个模块通常都有自己的 README。从那里开始了解设置和使用技巧。
|
||||
|
||||
### 如何报告错误或请求新功能？
|
||||
- [提交 GitHub 问题](https://github.com/microsoft/AI-For-Beginners/issues/new)，并提供清晰的描述和复现步骤。
|
||||
|
||||
### 如果我的问题未列出，可以寻求帮助吗？
|
||||
- 可以！先搜索现有问题，如果没有找到您的问题，请创建一个新问题。
|
||||
|
||||
---
|
||||
|
||||
## 获取帮助
|
||||
|
||||
- **检查问题：** [GitHub Issues](https://github.com/microsoft/AI-For-Beginners/issues)
|
||||
- **提问：** 使用 GitHub Discussions 或提交问题。
|
||||
- **社区：** 查看仓库链接以获取聊天/论坛选项。
|
||||
|
||||
---
|
||||
|
||||
_最后更新：2025-09-20_
|
||||
|
||||
---
|
||||
|
||||
**免责声明**：
|
||||
本文档使用AI翻译服务 [Co-op Translator](https://github.com/Azure/co-op-translator) 进行翻译。尽管我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。原始语言的文档应被视为权威来源。对于关键信息，建议使用专业人工翻译。我们对因使用此翻译而产生的任何误解或误读不承担责任。
|
||||
|
|
@ -0,0 +1,398 @@
|
|||
{
|
||||
"AGENTS.md": {
|
||||
"original_hash": "6b11a37115944252ab3ed04e358d830d",
|
||||
"translation_date": "2025-10-03T09:14:22+00:00",
|
||||
"source_file": "AGENTS.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"README.md": {
|
||||
"original_hash": "1984fc89dd304a8a33ab5584691a99aa",
|
||||
"translation_date": "2026-01-30T01:16:52+00:00",
|
||||
"source_file": "README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"SECURITY.md": {
|
||||
"original_hash": "a583f49d359c7ebba61433e4dfcd05a9",
|
||||
"translation_date": "2025-08-26T07:55:24+00:00",
|
||||
"source_file": "SECURITY.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"etc/CODE_OF_CONDUCT.md": {
|
||||
"original_hash": "c06b12caf3c901eb3156e3dd5b0aea56",
|
||||
"translation_date": "2025-08-26T11:17:02+00:00",
|
||||
"source_file": "etc/CODE_OF_CONDUCT.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"etc/CONTRIBUTING.md": {
|
||||
"original_hash": "847a587aa1b83f4d00858183ff3ed18a",
|
||||
"translation_date": "2025-08-26T11:19:34+00:00",
|
||||
"source_file": "etc/CONTRIBUTING.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"etc/Mindmap.md": {
|
||||
"original_hash": "f2f88dbd2debd38e26149b27b1fd272d",
|
||||
"translation_date": "2025-08-26T11:21:29+00:00",
|
||||
"source_file": "etc/Mindmap.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"etc/SUPPORT.md": {
|
||||
"original_hash": "fdfc08baee91e402938a2b1f94fe0949",
|
||||
"translation_date": "2025-08-26T11:16:09+00:00",
|
||||
"source_file": "etc/SUPPORT.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"etc/TRANSLATIONS.md": {
|
||||
"original_hash": "62b3e3ad5182edb905eec649a87eeeb4",
|
||||
"translation_date": "2025-08-26T11:18:08+00:00",
|
||||
"source_file": "etc/TRANSLATIONS.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"etc/quiz-app/README.md": {
|
||||
"original_hash": "d699cf8509f74baa5b0b838de5cf0662",
|
||||
"translation_date": "2025-08-26T11:25:13+00:00",
|
||||
"source_file": "etc/quiz-app/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"examples/README.md": {
|
||||
"original_hash": "0d1babfdcbeb46525f2db3fbaaa54cd7",
|
||||
"translation_date": "2025-10-03T11:27:40+00:00",
|
||||
"source_file": "examples/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/0-course-setup/for-teachers.md": {
|
||||
"original_hash": "a094ef9927883de1cfcee51dbd143381",
|
||||
"translation_date": "2025-08-26T11:11:56+00:00",
|
||||
"source_file": "lessons/0-course-setup/for-teachers.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/0-course-setup/how-to-run.md": {
|
||||
"original_hash": "a4717bd9103b9f6cd84d534b83534689",
|
||||
"translation_date": "2026-01-15T11:56:17+00:00",
|
||||
"source_file": "lessons/0-course-setup/how-to-run.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/0-course-setup/setup.md": {
|
||||
"original_hash": "7b4e5b8956915870d0a0ed3cc5890042",
|
||||
"translation_date": "2025-12-12T19:19:57+00:00",
|
||||
"source_file": "lessons/0-course-setup/setup.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/1-Intro/README.md": {
|
||||
"original_hash": "f57e8aa46141fd220b16ffed8f11aec7",
|
||||
"translation_date": "2025-11-18T20:49:11+00:00",
|
||||
"source_file": "lessons/1-Intro/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/1-Intro/assignment.md": {
|
||||
"original_hash": "a334df77a82aaaf2a29c77065d3e481e",
|
||||
"translation_date": "2025-11-18T20:50:47+00:00",
|
||||
"source_file": "lessons/1-Intro/assignment.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/2-Symbolic/README.md": {
|
||||
"original_hash": "f9f06b266b8b2bfc6b8792ff2bb1bea4",
|
||||
"translation_date": "2026-01-15T11:57:23+00:00",
|
||||
"source_file": "lessons/2-Symbolic/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/2-Symbolic/assignment.md": {
|
||||
"original_hash": "a057a8604f3976c3e309884453f1fad0",
|
||||
"translation_date": "2025-08-26T11:06:54+00:00",
|
||||
"source_file": "lessons/2-Symbolic/assignment.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/3-NeuralNetworks/03-Perceptron/README.md": {
|
||||
"original_hash": "c34cbba802058b6fa267e1a294d4e510",
|
||||
"translation_date": "2025-09-23T08:11:17+00:00",
|
||||
"source_file": "lessons/3-NeuralNetworks/03-Perceptron/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/3-NeuralNetworks/03-Perceptron/lab/README.md": {
|
||||
"original_hash": "ba5d1eb353d20d3e7181066b3c424b99",
|
||||
"translation_date": "2025-08-29T07:06:01+00:00",
|
||||
"source_file": "lessons/3-NeuralNetworks/03-Perceptron/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/3-NeuralNetworks/04-OwnFramework/README.md": {
|
||||
"original_hash": "789d6c3fb6fc7948a470b33078a5983a",
|
||||
"translation_date": "2025-09-23T08:10:56+00:00",
|
||||
"source_file": "lessons/3-NeuralNetworks/04-OwnFramework/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/3-NeuralNetworks/04-OwnFramework/lab/README.md": {
|
||||
"original_hash": "48fdd704d483e19bc3d7464074c9fcbe",
|
||||
"translation_date": "2025-08-26T10:29:08+00:00",
|
||||
"source_file": "lessons/3-NeuralNetworks/04-OwnFramework/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/3-NeuralNetworks/05-Frameworks/README.md": {
|
||||
"original_hash": "ddd216f558a255260a9374008002c971",
|
||||
"translation_date": "2025-09-23T08:11:38+00:00",
|
||||
"source_file": "lessons/3-NeuralNetworks/05-Frameworks/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/3-NeuralNetworks/05-Frameworks/lab/README.md": {
|
||||
"original_hash": "e452d897efb9a89700f41021834cf6e5",
|
||||
"translation_date": "2025-08-26T10:35:08+00:00",
|
||||
"source_file": "lessons/3-NeuralNetworks/05-Frameworks/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/3-NeuralNetworks/README.md": {
|
||||
"original_hash": "f862a99d88088163df12270e2f2ad6c3",
|
||||
"translation_date": "2025-10-03T12:43:12+00:00",
|
||||
"source_file": "lessons/3-NeuralNetworks/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/06-IntroCV/README.md": {
|
||||
"original_hash": "feeca98225cb420afc89415f24f63d92",
|
||||
"translation_date": "2025-09-23T08:07:00+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/06-IntroCV/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/06-IntroCV/lab/README.md": {
|
||||
"original_hash": "3d53d6409f80970f7281a45dee35328a",
|
||||
"translation_date": "2025-08-26T09:40:48+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/06-IntroCV/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/07-ConvNets/CNN_Architectures.md": {
|
||||
"original_hash": "53faab85adfcebd8c10bcd71dc2fa557",
|
||||
"translation_date": "2025-09-23T08:06:23+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/07-ConvNets/CNN_Architectures.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/07-ConvNets/README.md": {
|
||||
"original_hash": "a560d5b845962cf33dc102266e409568",
|
||||
"translation_date": "2025-09-23T08:06:06+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/07-ConvNets/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/07-ConvNets/lab/README.md": {
|
||||
"original_hash": "b70fcf7fcee862990f848c679090943f",
|
||||
"translation_date": "2025-10-03T14:52:50+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/07-ConvNets/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/08-TransferLearning/README.md": {
|
||||
"original_hash": "178c0b5ee5395733eb18aec51e71a0a9",
|
||||
"translation_date": "2025-09-23T08:06:37+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/08-TransferLearning/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/08-TransferLearning/TrainingTricks.md": {
|
||||
"original_hash": "ae074cd940fc2f4dc24fc07b66ccbd99",
|
||||
"translation_date": "2025-08-26T09:47:22+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/08-TransferLearning/TrainingTricks.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/08-TransferLearning/lab/README.md": {
|
||||
"original_hash": "7765935c35fcee69b9fe2d0cfd6963e2",
|
||||
"translation_date": "2025-08-26T09:52:46+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/08-TransferLearning/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/09-Autoencoders/README.md": {
|
||||
"original_hash": "1b8d9e1b3a6f1daa864b1ff3dfc3076d",
|
||||
"translation_date": "2025-09-23T08:08:07+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/09-Autoencoders/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/10-GANs/README.md": {
|
||||
"original_hash": "0ff65b4da07b23697235de2beb2a3c25",
|
||||
"translation_date": "2025-09-23T08:08:45+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/10-GANs/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/11-ObjectDetection/README.md": {
|
||||
"original_hash": "d76a7eda28de5210c8b1ba50a6216c69",
|
||||
"translation_date": "2025-09-23T08:07:28+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/11-ObjectDetection/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/11-ObjectDetection/lab/README.md": {
|
||||
"original_hash": "ad568d55ae65c856fe929fc2b278510a",
|
||||
"translation_date": "2025-08-26T09:24:51+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/11-ObjectDetection/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/12-Segmentation/README.md": {
|
||||
"original_hash": "6568aaae7e0e4afed4b5d74b5b223700",
|
||||
"translation_date": "2025-09-23T08:08:30+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/12-Segmentation/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/12-Segmentation/lab/README.md": {
|
||||
"original_hash": "365f0decfe0f47b460bbde8227c5009d",
|
||||
"translation_date": "2025-08-26T09:12:11+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/12-Segmentation/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/4-ComputerVision/README.md": {
|
||||
"original_hash": "58a52f000089c1d8906a4daa4ab1169b",
|
||||
"translation_date": "2025-08-26T09:02:40+00:00",
|
||||
"source_file": "lessons/4-ComputerVision/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/13-TextRep/README.md": {
|
||||
"original_hash": "dbd3f73e4139f030ecb2e20387d70fee",
|
||||
"translation_date": "2025-09-23T08:14:51+00:00",
|
||||
"source_file": "lessons/5-NLP/13-TextRep/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/13-TextRep/assignment.md": {
|
||||
"original_hash": "cdc1f2e631f055f3473b36d18e4760b3",
|
||||
"translation_date": "2025-08-26T08:28:32+00:00",
|
||||
"source_file": "lessons/5-NLP/13-TextRep/assignment.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/14-Embeddings/README.md": {
|
||||
"original_hash": "b708c9b85b833864c73c6281f1e6b96e",
|
||||
"translation_date": "2025-09-23T08:14:16+00:00",
|
||||
"source_file": "lessons/5-NLP/14-Embeddings/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/14-Embeddings/assignment.md": {
|
||||
"original_hash": "bc690ecf68b38d311cc9e12f3144a28c",
|
||||
"translation_date": "2025-08-26T08:17:44+00:00",
|
||||
"source_file": "lessons/5-NLP/14-Embeddings/assignment.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/15-LanguageModeling/README.md": {
|
||||
"original_hash": "7ba20f54a5bfcd6521018cdfb17c7c57",
|
||||
"translation_date": "2025-09-23T08:12:23+00:00",
|
||||
"source_file": "lessons/5-NLP/15-LanguageModeling/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/15-LanguageModeling/lab/README.md": {
|
||||
"original_hash": "5130f01fdc5ebb83032b23d489027aac",
|
||||
"translation_date": "2025-08-26T08:31:50+00:00",
|
||||
"source_file": "lessons/5-NLP/15-LanguageModeling/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/16-RNN/README.md": {
|
||||
"original_hash": "e2273cc150380a5e191903cea858f021",
|
||||
"translation_date": "2025-09-23T08:13:48+00:00",
|
||||
"source_file": "lessons/5-NLP/16-RNN/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/16-RNN/assignment.md": {
|
||||
"original_hash": "47f7d3c6a5373543e051e4d1140ce898",
|
||||
"translation_date": "2025-08-26T08:12:51+00:00",
|
||||
"source_file": "lessons/5-NLP/16-RNN/assignment.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/17-GenerativeNetworks/README.md": {
|
||||
"original_hash": "51be6057374d01d70e07dd5ec88ebc0d",
|
||||
"translation_date": "2025-09-23T08:12:05+00:00",
|
||||
"source_file": "lessons/5-NLP/17-GenerativeNetworks/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/17-GenerativeNetworks/lab/README.md": {
|
||||
"original_hash": "439e12796197a90e7623d4c9c057b9c2",
|
||||
"translation_date": "2025-08-26T08:22:51+00:00",
|
||||
"source_file": "lessons/5-NLP/17-GenerativeNetworks/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/18-Transformers/README.md": {
|
||||
"original_hash": "f335dfcb4a993920504c387973a36957",
|
||||
"translation_date": "2025-09-23T08:12:37+00:00",
|
||||
"source_file": "lessons/5-NLP/18-Transformers/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/18-Transformers/assignment.md": {
|
||||
"original_hash": "177f3ea3995d725e6f9f5c66af16edcd",
|
||||
"translation_date": "2025-08-26T08:40:43+00:00",
|
||||
"source_file": "lessons/5-NLP/18-Transformers/assignment.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/19-NER/README.md": {
|
||||
"original_hash": "6522312ff835796ca34136a9462fafb2",
|
||||
"translation_date": "2025-09-23T08:13:29+00:00",
|
||||
"source_file": "lessons/5-NLP/19-NER/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/19-NER/lab/README.md": {
|
||||
"original_hash": "032bda5068f543d6c1fcb30c34231461",
|
||||
"translation_date": "2025-08-26T08:50:12+00:00",
|
||||
"source_file": "lessons/5-NLP/19-NER/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/20-LangModels/README.md": {
|
||||
"original_hash": "97836d30a6bec736f8e3b4411c572bc2",
|
||||
"translation_date": "2025-09-23T08:13:06+00:00",
|
||||
"source_file": "lessons/5-NLP/20-LangModels/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/5-NLP/README.md": {
|
||||
"original_hash": "8ef02a9318257ea140ed3ed74442096d",
|
||||
"translation_date": "2025-08-26T08:02:59+00:00",
|
||||
"source_file": "lessons/5-NLP/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/6-Other/21-GeneticAlgorithms/README.md": {
|
||||
"original_hash": "6bbd632dfe6c62e5f66bb51fd78c174a",
|
||||
"translation_date": "2025-09-23T08:04:56+00:00",
|
||||
"source_file": "lessons/6-Other/21-GeneticAlgorithms/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/6-Other/22-DeepRL/README.md": {
|
||||
"original_hash": "04395657fc01648f8f70484d0e55ab67",
|
||||
"translation_date": "2025-09-23T08:05:44+00:00",
|
||||
"source_file": "lessons/6-Other/22-DeepRL/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/6-Other/22-DeepRL/lab/README.md": {
|
||||
"original_hash": "7bd8dc72040e98e35e7225e34058cd4e",
|
||||
"translation_date": "2025-08-26T10:16:40+00:00",
|
||||
"source_file": "lessons/6-Other/22-DeepRL/lab/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/6-Other/23-MultiagentSystems/README.md": {
|
||||
"original_hash": "38a1185ae3d54b180378bbd71ae3ef16",
|
||||
"translation_date": "2025-09-23T08:05:09+00:00",
|
||||
"source_file": "lessons/6-Other/23-MultiagentSystems/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/6-Other/23-MultiagentSystems/assignment.md": {
|
||||
"original_hash": "cf654ca60c7f86c8dad28596fb42994b",
|
||||
"translation_date": "2025-08-26T10:11:06+00:00",
|
||||
"source_file": "lessons/6-Other/23-MultiagentSystems/assignment.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/7-Ethics/README.md": {
|
||||
"original_hash": "437c988596e751072e41a5aad3fcc5d9",
|
||||
"translation_date": "2025-08-26T07:58:36+00:00",
|
||||
"source_file": "lessons/7-Ethics/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/README.md": {
|
||||
"original_hash": "5fef1a0b22498d7188959e2a2cb08af7",
|
||||
"translation_date": "2025-08-26T07:57:15+00:00",
|
||||
"source_file": "lessons/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/X-Extras/X1-MultiModal/README.md": {
|
||||
"original_hash": "9c592c26aca16ca085d268c732284187",
|
||||
"translation_date": "2025-08-26T10:18:27+00:00",
|
||||
"source_file": "lessons/X-Extras/X1-MultiModal/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/sketchnotes/LICENSE.md": {
|
||||
"original_hash": "45ab63a2cd8f5faef6c9b150618837a4",
|
||||
"translation_date": "2025-08-26T10:44:54+00:00",
|
||||
"source_file": "lessons/sketchnotes/LICENSE.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"lessons/sketchnotes/README.md": {
|
||||
"original_hash": "050b8bddebafba55b129414e6ab096ab",
|
||||
"translation_date": "2025-08-26T10:40:32+00:00",
|
||||
"source_file": "lessons/sketchnotes/README.md",
|
||||
"language_code": "zh-MO"
|
||||
},
|
||||
"troubleshoot.md": {
|
||||
"original_hash": "8d9c5a4a7c7798d699672a22cb7fea86",
|
||||
"translation_date": "2025-10-03T09:37:05+00:00",
|
||||
"source_file": "troubleshoot.md",
|
||||
"language_code": "zh-MO"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,317 @@
|
|||
# AGENTS.md
|
||||
|
||||
## 專案概述
|
||||
|
||||
AI for Beginners 是一個為期 12 週、共 24 節課的全面課程，涵蓋人工智慧的基礎知識。本教育資源庫包含使用 Jupyter Notebooks 的實作課程、小測驗以及實作實驗室。課程內容包括：
|
||||
|
||||
- 使用知識表示和專家系統的符號 AI
|
||||
- 使用 TensorFlow 和 PyTorch 的神經網路與深度學習
|
||||
- 電腦視覺技術與架構
|
||||
- 自然語言處理 (NLP)，包括 transformers 和 BERT
|
||||
- 專題：遺傳演算法、強化學習、多代理系統
|
||||
- AI 倫理與負責任的 AI 原則
|
||||
|
||||
**主要技術：** Python 3、Jupyter Notebooks、TensorFlow、PyTorch、Keras、OpenCV、Vue.js（用於測驗應用程式）
|
||||
|
||||
**架構：** 教育內容資源庫，按主題區域組織的 Jupyter Notebooks，輔以基於 Vue.js 的測驗應用程式以及廣泛的多語言支援。
|
||||
|
||||
## 設置指令
|
||||
|
||||
### 主要開發環境 (Python/Jupyter)
|
||||
|
||||
課程設計基於 Python 和 Jupyter Notebooks 運行。推薦使用 miniconda：
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/microsoft/ai-for-beginners
|
||||
cd ai-for-beginners
|
||||
|
||||
# Create and activate conda environment
|
||||
conda env create --name ai4beg --file environment.yml
|
||||
conda activate ai4beg
|
||||
|
||||
# Start Jupyter Notebook
|
||||
jupyter notebook
|
||||
# OR
|
||||
jupyter lab
|
||||
```
|
||||
|
||||
### 替代方案：使用 devcontainer
|
||||
|
||||
```bash
|
||||
# Open in VS Code and select "Reopen in Container" when prompted
|
||||
# The devcontainer will automatically set up the environment
|
||||
```
|
||||
|
||||
### 測驗應用程式設置
|
||||
|
||||
測驗應用程式是一個獨立的 Vue.js 應用程式，位於 `etc/quiz-app/`：
|
||||
|
||||
```bash
|
||||
cd etc/quiz-app
|
||||
npm install
|
||||
npm run serve # Development server
|
||||
npm run build # Production build
|
||||
npm run lint # Lint and fix files
|
||||
```
|
||||
|
||||
## 開發工作流程
|
||||
|
||||
### 使用 Jupyter Notebooks
|
||||
|
||||
1. **本地開發：**
|
||||
- 啟用 conda 環境：`conda activate ai4beg`
|
||||
- 啟動 Jupyter：`jupyter notebook` 或 `jupyter lab`
|
||||
- 瀏覽到課程資料夾並打開 `.ipynb` 文件
|
||||
- 互動式執行單元以跟隨課程
|
||||
|
||||
2. **使用 VS Code 和 Python 擴展：**
|
||||
- 在 VS Code 中打開資源庫
|
||||
- 安裝 Python 擴展
|
||||
- VS Code 會自動檢測並使用 conda 環境
|
||||
- 直接在 VS Code 中打開 `.ipynb` 文件
|
||||
|
||||
3. **雲端開發：**
|
||||
- **GitHub Codespaces：** 點擊 "Code" → "Codespaces" → "Create codespace on main"
|
||||
- **Binder：** 使用 README 中的 Binder 徽章在瀏覽器中啟動
|
||||
- 注意：Binder 資源有限，且有部分網路訪問限制
|
||||
|
||||
### 高級課程的 GPU 支援
|
||||
|
||||
後期課程顯著受益於 GPU 加速：
|
||||
|
||||
- **Azure Data Science VM：** 使用支援 GPU 的 NC 系列虛擬機
|
||||
- **Azure Machine Learning：** 使用具備 GPU 計算的筆記本功能
|
||||
- **Google Colab：** 單獨上傳筆記本（提供免費 GPU 支援）
|
||||
|
||||
### 測驗應用程式開發
|
||||
|
||||
```bash
|
||||
cd etc/quiz-app
|
||||
npm run serve # Hot-reload development server at http://localhost:8080
|
||||
```
|
||||
|
||||
## 測試說明
|
||||
|
||||
這是一個專注於學習內容的教育資源庫，而非軟體測試，因此沒有傳統的測試套件。
|
||||
|
||||
### 驗證方法：
|
||||
|
||||
1. **Jupyter Notebooks：** 按順序執行單元以驗證程式碼範例是否正常運行
|
||||
2. **測驗應用程式測試：** 通過開發伺服器進行手動測試
|
||||
3. **翻譯驗證：** 檢查 `translations/` 資料夾中的翻譯內容
|
||||
4. **測驗應用程式檢查：** 在 `etc/quiz-app/` 中執行 `npm run lint`
|
||||
|
||||
### 執行程式碼範例：
|
||||
|
||||
```bash
|
||||
# Activate environment first
|
||||
conda activate ai4beg
|
||||
|
||||
# Run Python scripts directly
|
||||
python lessons/4-ComputerVision/07-ConvNets/pytorchcv.py
|
||||
|
||||
# Or execute notebooks
|
||||
jupyter notebook lessons/3-NeuralNetworks/03-Perceptron/Perceptron.ipynb
|
||||
```
|
||||
|
||||
## 程式碼風格
|
||||
|
||||
### Python 程式碼風格
|
||||
|
||||
- 遵循教育程式碼的標準 Python 慣例
|
||||
- 優先考慮學習的清晰、可讀性程式碼，而非最佳化
|
||||
- 添加註解以解釋關鍵概念
|
||||
- 適合 Jupyter Notebook：單元應儘可能自包含
|
||||
- 對課程內容無嚴格的程式碼檢查要求
|
||||
|
||||
### JavaScript/Vue.js（測驗應用程式）
|
||||
|
||||
- ESLint 配置位於 `etc/quiz-app/package.json`
|
||||
- 執行 `npm run lint` 以檢查並自動修復問題
|
||||
- 遵循 Vue 2.x 慣例
|
||||
- 基於元件的架構
|
||||
|
||||
### 文件組織
|
||||
|
||||
```
|
||||
lessons/
|
||||
├── 0-course-setup/ # Setup instructions
|
||||
├── 1-Intro/ # Introduction to AI
|
||||
├── 2-Symbolic/ # Symbolic AI
|
||||
├── 3-NeuralNetworks/ # Neural Networks basics
|
||||
├── 4-ComputerVision/ # Computer Vision
|
||||
├── 5-NLP/ # Natural Language Processing
|
||||
├── 6-Other/ # Other AI techniques
|
||||
├── 7-Ethics/ # AI Ethics
|
||||
└── X-Extras/ # Additional content
|
||||
|
||||
etc/
|
||||
├── quiz-app/ # Vue.js quiz application
|
||||
└── quiz-src/ # Quiz source files
|
||||
|
||||
translations/ # Multi-language translations
|
||||
```
|
||||
|
||||
## 構建與部署
|
||||
|
||||
### Jupyter 內容
|
||||
|
||||
無需構建過程 - Jupyter Notebooks 可直接執行。
|
||||
|
||||
### 測驗應用程式
|
||||
|
||||
```bash
|
||||
cd etc/quiz-app
|
||||
|
||||
# Development
|
||||
npm run serve
|
||||
|
||||
# Production build
|
||||
npm run build # Outputs to etc/quiz-app/dist/
|
||||
|
||||
# Deploy to Azure Static Web Apps
|
||||
# Azure automatically creates GitHub Actions workflow
|
||||
# See etc/quiz-app/README.md for detailed deployment instructions
|
||||
```
|
||||
|
||||
### 文件網站
|
||||
|
||||
資源庫使用 Docsify 作為文件系統：
|
||||
- `index.html` 作為入口點
|
||||
- 無需構建 - 直接通過 GitHub Pages 提供服務
|
||||
- 訪問地址：https://microsoft.github.io/AI-For-Beginners/
|
||||
|
||||
## 貢獻指南
|
||||
|
||||
### 提交請求流程
|
||||
|
||||
1. **標題格式：** 清晰、描述性的標題，說明更改內容
|
||||
2. **CLA 要求：** 必須簽署 Microsoft CLA（自動檢查）
|
||||
3. **內容指南：**
|
||||
- 保持教育重點和適合初學者的方式
|
||||
- 測試筆記本中的所有程式碼範例
|
||||
- 確保筆記本能從頭到尾運行
|
||||
- 如果修改英文內容，請更新翻譯
|
||||
4. **測驗應用程式更改：** 提交前執行 `npm run lint`
|
||||
|
||||
### 翻譯貢獻
|
||||
|
||||
- 翻譯通過 GitHub Actions 使用 co-op-translator 自動完成
|
||||
- 手動翻譯存放於 `translations/<language-code>/`
|
||||
- 測驗翻譯存放於 `etc/quiz-app/src/assets/translations/`
|
||||
- 支援語言：40+ 種語言（完整列表見 README）
|
||||
|
||||
### 活躍貢獻領域
|
||||
|
||||
請參閱 `etc/CONTRIBUTING.md` 了解當前需求：
|
||||
- 深度強化學習部分
|
||||
- 物件檢測改進
|
||||
- 命名實體識別範例
|
||||
- 自定義嵌入訓練範例
|
||||
|
||||
## 環境配置
|
||||
|
||||
### 必需的依賴項
|
||||
|
||||
```bash
|
||||
# Core Python packages (from requirements.txt)
|
||||
tensorflow==2.17.0
|
||||
torch (via conda)
|
||||
torchvision (via conda)
|
||||
keras==3.5.0
|
||||
opencv (via conda)
|
||||
scikit-learn
|
||||
numpy==1.26
|
||||
pandas==2.2.2
|
||||
matplotlib==3.9
|
||||
jupyter
|
||||
```
|
||||
|
||||
### 環境變數
|
||||
|
||||
基本使用無需特殊環境變數。
|
||||
|
||||
對於 Azure 部署（測驗應用程式）：
|
||||
- `AZURE_STATIC_WEB_APPS_API_TOKEN`（由 Azure 自動設置）
|
||||
|
||||
## 偵錯與故障排除
|
||||
|
||||
### 常見問題
|
||||
|
||||
**問題：** Conda 環境創建失敗
|
||||
- **解決方案：** 先更新 conda：`conda update conda -y`
|
||||
- 確保有足夠的磁碟空間（建議 50GB）
|
||||
|
||||
**問題：** 找不到 Jupyter 核心
|
||||
- **解決方案：**
|
||||
```bash
|
||||
conda activate ai4beg
|
||||
python -m ipykernel install --user --name ai4beg
|
||||
```
|
||||
|
||||
**問題：** 筆記本中未檢測到 GPU
|
||||
- **解決方案：**
|
||||
- 驗證 CUDA 安裝：`nvidia-smi`
|
||||
- 檢查 PyTorch GPU：`python -c "import torch; print(torch.cuda.is_available())"`
|
||||
- 檢查 TensorFlow GPU：`python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"`
|
||||
|
||||
**問題：** 測驗應用程式無法啟動
|
||||
- **解決方案：**
|
||||
```bash
|
||||
cd etc/quiz-app
|
||||
rm -rf node_modules package-lock.json
|
||||
npm install
|
||||
npm run serve
|
||||
```
|
||||
|
||||
**問題：** Binder 超時或阻止下載
|
||||
- **解決方案：** 使用 GitHub Codespaces 或本地設置以獲得更好的資源訪問
|
||||
|
||||
### 記憶體問題
|
||||
|
||||
某些課程需要大量 RAM（建議 8GB+）：
|
||||
- 對於資源密集型課程，使用雲端虛擬機
|
||||
- 在訓練模型時關閉其他應用程式
|
||||
- 如果記憶體不足，減小筆記本中的批次大小
|
||||
|
||||
## 附加說明
|
||||
|
||||
### 對課程講師的建議
|
||||
|
||||
- 請參閱 `lessons/0-course-setup/for-teachers.md` 獲取教學指導
|
||||
- 課程是自包含的，可以按順序教學或單獨選擇
|
||||
- 預估時間：每週 2 節課，共 12 週
|
||||
|
||||
### 雲端資源
|
||||
|
||||
- **Azure for Students：** 學生可獲得免費額度
|
||||
- **Microsoft Learn：** 課程中鏈接的補充學習路徑
|
||||
- **Binder：** 免費但資源有限，且有部分網路限制
|
||||
|
||||
### 程式碼執行選項
|
||||
|
||||
1. **本地（推薦）：** 完全控制，最佳效能，支援 GPU
|
||||
2. **GitHub Codespaces：** 基於雲端的 VS Code，快速訪問
|
||||
3. **Binder：** 基於瀏覽器的 Jupyter，免費但有限制
|
||||
4. **Azure ML Notebooks：** 企業級選項，支援 GPU
|
||||
5. **Google Colab：** 單獨上傳筆記本，提供免費 GPU 層
|
||||
|
||||
### 使用筆記本
|
||||
|
||||
- 筆記本設計為逐單元運行以便學習
|
||||
- 許多筆記本在首次運行時會下載數據集（可能需要一些時間）
|
||||
- 某些模型需要 GPU 才能在合理時間內完成訓練
|
||||
- 儘可能使用預訓練模型以減少計算需求
|
||||
|
||||
### 效能考量
|
||||
|
||||
- 後期的電腦視覺課程（CNNs、GANs）受益於 GPU
|
||||
- NLP transformer 課程可能需要大量 RAM
|
||||
- 從頭開始訓練具有教育意義，但耗時
|
||||
- 遷移學習範例可最大限度減少訓練時間
|
||||
|
||||
---
|
||||
|
||||
**免責聲明**：
|
||||
本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。雖然我們致力於提供準確的翻譯，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵資訊，建議使用專業人工翻譯。我們對因使用此翻譯而產生的任何誤解或錯誤解釋不承擔責任。
|
||||
|
|
@ -0,0 +1,224 @@
|
|||
[](https://github.com/microsoft/AI-For-Beginners/blob/main/LICENSE)
|
||||
[](https://GitHub.com/microsoft/AI-For-Beginners/graphs/contributors/)
|
||||
[](https://GitHub.com/microsoft/AI-For-Beginners/issues/)
|
||||
[](https://GitHub.com/microsoft/AI-For-Beginners/pulls/)
|
||||
[](http://makeapullrequest.com)
|
||||
|
||||
[](https://GitHub.com/microsoft/AI-For-Beginners/watchers/)
|
||||
[](https://GitHub.com/microsoft/AI-For-Beginners/network/)
|
||||
[](https://GitHub.com/microsoft/AI-For-Beginners/stargazers/)
|
||||
[](https://mybinder.org/v2/gh/microsoft/ai-for-beginners/HEAD)
|
||||
[](https://gitter.im/Microsoft/ai-for-beginners?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
|
||||
|
||||
[](https://discord.gg/nTYy5BXMWG)
|
||||
|
||||
# 初學者人工智能課程
|
||||
|
||||
||
|
||||
|:---:|
|
||||
| AI For Beginners - _草圖筆記由 [@girlie_mac](https://twitter.com/girlie_mac) 製作_ |
|
||||
|
||||
透過我們為期12週、共24課的課程，探索**人工智能**（AI）的世界！課程包括實作課程、小測驗和實驗室單元。此課程對初學者友善，涵蓋了TensorFlow和PyTorch等工具，並探討AI倫理。
|
||||
|
||||
### 🌐 多語言支援
|
||||
|
||||
#### 透過 GitHub Action 支援（自動化且持續更新）
|
||||
|
||||
<!-- CO-OP TRANSLATOR LANGUAGES TABLE START -->
|
||||
[Arabic](../ar/README.md) | [Bengali](../bn/README.md) | [Bulgarian](../bg/README.md) | [Burmese (Myanmar)](../my/README.md) | [Chinese (Simplified)](../zh-CN/README.md) | [Chinese (Traditional, Hong Kong)](../zh-HK/README.md) | [Chinese (Traditional, Macau)](./README.md) | [Chinese (Traditional, Taiwan)](../zh-TW/README.md) | [Croatian](../hr/README.md) | [Czech](../cs/README.md) | [Danish](../da/README.md) | [Dutch](../nl/README.md) | [Estonian](../et/README.md) | [Finnish](../fi/README.md) | [French](../fr/README.md) | [German](../de/README.md) | [Greek](../el/README.md) | [Hebrew](../he/README.md) | [Hindi](../hi/README.md) | [Hungarian](../hu/README.md) | [Indonesian](../id/README.md) | [Italian](../it/README.md) | [Japanese](../ja/README.md) | [Kannada](../kn/README.md) | [Korean](../ko/README.md) | [Lithuanian](../lt/README.md) | [Malay](../ms/README.md) | [Malayalam](../ml/README.md) | [Marathi](../mr/README.md) | [Nepali](../ne/README.md) | [Nigerian Pidgin](../pcm/README.md) | [Norwegian](../no/README.md) | [Persian (Farsi)](../fa/README.md) | [Polish](../pl/README.md) | [Portuguese (Brazil)](../pt-BR/README.md) | [Portuguese (Portugal)](../pt-PT/README.md) | [Punjabi (Gurmukhi)](../pa/README.md) | [Romanian](../ro/README.md) | [Russian](../ru/README.md) | [Serbian (Cyrillic)](../sr/README.md) | [Slovak](../sk/README.md) | [Slovenian](../sl/README.md) | [Spanish](../es/README.md) | [Swahili](../sw/README.md) | [Swedish](../sv/README.md) | [Tagalog (Filipino)](../tl/README.md) | [Tamil](../ta/README.md) | [Telugu](../te/README.md) | [Thai](../th/README.md) | [Turkish](../tr/README.md) | [Ukrainian](../uk/README.md) | [Urdu](../ur/README.md) | [Vietnamese](../vi/README.md)
|
||||
|
||||
> **偏好本地複製？**
|
||||
|
||||
> 此存儲庫包含50多種語言的翻譯，會顯著增加下載大小。如需不包含翻譯的複製，請使用稀疏檢出：
|
||||
> ```bash
|
||||
> git clone --filter=blob:none --sparse https://github.com/microsoft/AI-For-Beginners.git
|
||||
> cd AI-For-Beginners
|
||||
> git sparse-checkout set --no-cone '/*' '!translations' '!translated_images'
|
||||
> ```
|
||||
> 這樣您便能以更快的速度下載並擁有完成課程所需的所有資源。
|
||||
<!-- CO-OP TRANSLATOR LANGUAGES TABLE END -->
|
||||
|
||||
**如需新增翻譯語言支援，請參考[這裡](https://github.com/Azure/co-op-translator/blob/main/getting_started/supported-languages.md)**
|
||||
|
||||
## 加入社群
|
||||
[](https://discord.gg/nTYy5BXMWG)
|
||||
|
||||
## 您將學到什麼
|
||||
|
||||
**[課程心智圖](http://soshnikov.com/courses/ai-for-beginners/mindmap.html)**
|
||||
|
||||
在此課程中，您將學習：
|
||||
|
||||
* 各種人工智能方法，包括「經典的」符號方法，涵蓋**知識表示**與推理（[GOFAI](https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence)）。
|
||||
* **神經網絡**與**深度學習**，這是現代AI的核心。我們將利用目前最流行的兩個框架—[TensorFlow](http://Tensorflow.org) 和 [PyTorch](http://pytorch.org) 的程式碼說明這些重要主題背後的概念。
|
||||
* 運用於影像和文本的**神經架構**。我們會涵蓋一些近期的模型，但可能在最先進技術方面有所不足。
|
||||
* 較不普及的AI方法，如**遺傳演算法**與**多智能體系統**。
|
||||
|
||||
本課程不涵蓋的內容：
|
||||
|
||||
> [在我們的 Microsoft Learn 集合中找到此課程所有額外資源](https://learn.microsoft.com/en-us/collections/7w28iy2xrqzdj0?WT.mc_id=academic-77998-bethanycheum)
|
||||
|
||||
* **商業中的AI案例**。建議您可參加 Microsoft Learn 上的 [面向商業用戶的AI入門](https://docs.microsoft.com/learn/paths/introduction-ai-for-business-users/?WT.mc_id=academic-77998-bethanycheum) 學習路徑，或與 [INSEAD](https://www.insead.edu/) 合作開發的 [AI 商業學院](https://www.microsoft.com/ai/ai-business-school/?WT.mc_id=academic-77998-bethanycheum)。
|
||||
* **經典機器學習**，詳見我們的 [初學者機器學習課程](http://github.com/Microsoft/ML-for-Beginners)。
|
||||
* 利用 **[認知服務](https://azure.microsoft.com/services/cognitive-services/?WT.mc_id=academic-77998-bethanycheum)** 建構的實務AI應用。我們推薦您先從 Microsoft Learn 中的 [視覺](https://docs.microsoft.com/learn/paths/create-computer-vision-solutions-azure-cognitive-services/?WT.mc_id=academic-77998-bethanycheum) 和 [自然語言處理](https://docs.microsoft.com/learn/paths/explore-natural-language-processing/?WT.mc_id=academic-77998-bethanycheum)、**[Azure OpenAI 服務的生成式AI](https://learn.microsoft.com/en-us/training/paths/develop-ai-solutions-azure-openai/?WT.mc_id=academic-77998-bethanycheum)** 相關課程開始。
|
||||
* 特定的機器學習**雲端框架**，例如 [Azure 機器學習](https://azure.microsoft.com/services/machine-learning/?WT.mc_id=academic-77998-bethanycheum)、[Microsoft Fabric](https://learn.microsoft.com/en-us/training/paths/get-started-fabric/?WT.mc_id=academic-77998-bethanycheum) 或 [Azure Databricks](https://docs.microsoft.com/learn/paths/data-engineer-azure-databricks?WT.mc_id=academic-77998-bethanycheum)。建議參閱 "使用 Azure 機器學習建立與管理機器學習解決方案" 與 "使用 Azure Databricks 建立與管理機器學習解決方案" 學習路徑。
|
||||
* **對話式AI**與**聊天機器人**。另外有獨立的 [建立對話式AI解決方案](https://docs.microsoft.com/learn/paths/create-conversational-ai-solutions/?WT.mc_id=academic-77998-bethanycheum) 學習路徑，亦可參考 [此博客文章](https://soshnikov.com/azure/hello-bot-conversational-ai-on-microsoft-platform/) 獲取更多詳情。
|
||||
* 深度學習背後的**深度數學**。對此我們推薦 Ian Goodfellow、Yoshua Bengio 與 Aaron Courville 的著作 [Deep Learning](https://www.amazon.com/Deep-Learning-Adaptive-Computation-Machine/dp/0262035618)，該書也可於線上免費取得 [https://www.deeplearningbook.org/](https://www.deeplearningbook.org/)。
|
||||
|
||||
若想輕鬆入門「雲端AI」主題，您可以考慮修讀 [Azure上的人工智能入門](https://docs.microsoft.com/learn/paths/get-started-with-artificial-intelligence-on-azure/?WT.mc_id=academic-77998-bethanycheum) 學習路徑。
|
||||
|
||||
# 內容
|
||||
|
||||
| | 課程連結 | PyTorch/Keras/TensorFlow | 實驗室 |
|
||||
| :-: | :------------------------------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------: | ------------------------------------------------------------------------------ |
|
||||
| 0 | [課程設定](./lessons/0-course-setup/setup.md) | [設定您的開發環境](./lessons/0-course-setup/how-to-run.md) | |
|
||||
| I | [**AI 簡介**](./lessons/1-Intro/README.md) | | |
|
||||
| 01 | [人工智能簡介與歷史](./lessons/1-Intro/README.md) | - | - |
|
||||
| II | **符號式AI** |
|
||||
| 02 | [知識表示與專家系統](./lessons/2-Symbolic/README.md) | [專家系統](./lessons/2-Symbolic/Animals.ipynb) / [本體論](./lessons/2-Symbolic/FamilyOntology.ipynb) /[概念圖](./lessons/2-Symbolic/MSConceptGraph.ipynb) | |
|
||||
| III | [**神經網絡簡介**](./lessons/3-NeuralNetworks/README.md) |||
|
||||
| 03 | [感知機](./lessons/3-NeuralNetworks/03-Perceptron/README.md) | [筆記本](./lessons/3-NeuralNetworks/03-Perceptron/Perceptron.ipynb) | [實驗室](./lessons/3-NeuralNetworks/03-Perceptron/lab/README.md) |
|
||||
| 04 | [多層感知機及自建框架](./lessons/3-NeuralNetworks/04-OwnFramework/README.md) | [筆記本](./lessons/3-NeuralNetworks/04-OwnFramework/OwnFramework.ipynb) | [實驗室](./lessons/3-NeuralNetworks/04-OwnFramework/lab/README.md) |
|
||||
| 05 | [框架介紹（PyTorch/TensorFlow）與過擬合](./lessons/3-NeuralNetworks/05-Frameworks/README.md) | [PyTorch](./lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb) / [Keras](./lessons/3-NeuralNetworks/05-Frameworks/IntroKeras.ipynb) / [TensorFlow](./lessons/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb) | [實驗室](./lessons/3-NeuralNetworks/05-Frameworks/lab/README.md) |
|
||||
| IV | [**電腦視覺**](./lessons/4-ComputerVision/README.md) | [PyTorch](https://docs.microsoft.com/learn/modules/intro-computer-vision-pytorch/?WT.mc_id=academic-77998-cacaste) / [TensorFlow](https://docs.microsoft.com/learn/modules/intro-computer-vision-TensorFlow/?WT.mc_id=academic-77998-cacaste)| [在 Microsoft Azure 探索電腦視覺](https://learn.microsoft.com/en-us/collections/7w28iy2xrqzdj0?WT.mc_id=academic-77998-bethanycheum) |
|
||||
| 06 | [電腦視覺入門。OpenCV](./lessons/4-ComputerVision/06-IntroCV/README.md) | [筆記本](./lessons/4-ComputerVision/06-IntroCV/OpenCV.ipynb) | [實驗室](./lessons/4-ComputerVision/06-IntroCV/lab/README.md) |
|
||||
| 07 | [卷積神經網絡](./lessons/4-ComputerVision/07-ConvNets/README.md) & [CNN 架構](./lessons/4-ComputerVision/07-ConvNets/CNN_Architectures.md) | [PyTorch](./lessons/4-ComputerVision/07-ConvNets/ConvNetsPyTorch.ipynb) /[TensorFlow](./lessons/4-ComputerVision/07-ConvNets/ConvNetsTF.ipynb) | [實驗室](./lessons/4-ComputerVision/07-ConvNets/lab/README.md) |
|
||||
| 08 | [預訓練網絡與遷移學習](./lessons/4-ComputerVision/08-TransferLearning/README.md) 和 [訓練技巧](./lessons/4-ComputerVision/08-TransferLearning/TrainingTricks.md) | [PyTorch](./lessons/4-ComputerVision/08-TransferLearning/TransferLearningPyTorch.ipynb) / [TensorFlow](./lessons/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb) | [實驗室](./lessons/4-ComputerVision/08-TransferLearning/lab/README.md) |
|
||||
| 09 | [自編碼器與變分自編碼器](./lessons/4-ComputerVision/09-Autoencoders/README.md) | [PyTorch](./lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb) / [TensorFlow](./lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb) | |
|
||||
| 10 | [生成對抗網絡與藝術風格轉換](./lessons/4-ComputerVision/10-GANs/README.md) | [PyTorch](./lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb) / [TensorFlow](./lessons/4-ComputerVision/10-GANs/GANTF.ipynb) | |
|
||||
| 11 | [物件偵測](./lessons/4-ComputerVision/11-ObjectDetection/README.md) | [TensorFlow](./lessons/4-ComputerVision/11-ObjectDetection/ObjectDetection.ipynb) | [實驗室](./lessons/4-ComputerVision/11-ObjectDetection/lab/README.md) |
|
||||
| 12 | [語義分割。U-Net](./lessons/4-ComputerVision/12-Segmentation/README.md) | [PyTorch](./lessons/4-ComputerVision/12-Segmentation/SemanticSegmentationPytorch.ipynb) / [TensorFlow](./lessons/4-ComputerVision/12-Segmentation/SemanticSegmentationTF.ipynb) | |
|
||||
| V | [**自然語言處理**](./lessons/5-NLP/README.md) | [PyTorch](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-pytorch/?WT.mc_id=academic-77998-cacaste) /[TensorFlow](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-TensorFlow/?WT.mc_id=academic-77998-cacaste) | [在 Microsoft Azure 探索自然語言處理](https://learn.microsoft.com/en-us/collections/7w28iy2xrqzdj0?WT.mc_id=academic-77998-bethanycheum)|
|
||||
| 13 | [文本表示。詞袋模型/TF-IDF](./lessons/5-NLP/13-TextRep/README.md) | [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/13-TextRep/TextRepresentationPyTorch.ipynb) / [TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/13-TextRep/TextRepresentationTF.ipynb) | |
|
||||
| 14 | [語義詞嵌入。Word2Vec 與 GloVe](./lessons/5-NLP/14-Embeddings/README.md) | [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/14-Embeddings/EmbeddingsPyTorch.ipynb) / [TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/14-Embeddings/EmbeddingsTF.ipynb) | |
|
||||
| 15 | [語言模型。訓練你自己的詞嵌入](./lessons/5-NLP/15-LanguageModeling/README.md) | [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/15-LanguageModeling/CBoW-PyTorch.ipynb) / [TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/15-LanguageModeling/CBoW-TF.ipynb) | [實驗室](./lessons/5-NLP/15-LanguageModeling/lab/README.md) |
|
||||
| 16 | [循環神經網絡](./lessons/5-NLP/16-RNN/README.md) | [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/16-RNN/RNNPyTorch.ipynb) / [TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/16-RNN/RNNTF.ipynb) | |
|
||||
| 17 | [生成循環網絡](./lessons/5-NLP/17-GenerativeNetworks/README.md) | [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.ipynb) / [TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.ipynb) | [實驗室](./lessons/5-NLP/17-GenerativeNetworks/lab/README.md) |
|
||||
| 18 | [變壓器模型。BERT。](./lessons/5-NLP/18-Transformers/README.md) | [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/18-Transformers/TransformersPyTorch.ipynb) /[TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/18-Transformers/TransformersTF.ipynb) | |
|
||||
| 19 | [命名實體識別](./lessons/5-NLP/19-NER/README.md) | [TensorFlow](https://microsoft.github.io/AI-For-Beginners/lessons/5-NLP/19-NER/NER-TF.ipynb) | [實驗室](./lessons/5-NLP/19-NER/lab/README.md) |
|
||||
| 20 | [大型語言模型、提示編程與少量樣本任務](./lessons/5-NLP/20-LangModels/README.md) | [PyTorch](https://microsoft.github.io/AI-For-Beginners/lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb) | |
|
||||
| VI | **其他 AI 技術** || |
|
||||
| 21 | [基因演算法](./lessons/6-Other/21-GeneticAlgorithms/README.md) | [筆記本](./lessons/6-Other/21-GeneticAlgorithms/Genetic.ipynb) | |
|
||||
| 22 | [深度強化學習](./lessons/6-Other/22-DeepRL/README.md) | [PyTorch](./lessons/6-Other/22-DeepRL/CartPole-RL-PyTorch.ipynb) /[TensorFlow](./lessons/6-Other/22-DeepRL/CartPole-RL-TF.ipynb) | [實驗室](./lessons/6-Other/22-DeepRL/lab/README.md) |
|
||||
| 23 | [多智能體系統](./lessons/6-Other/23-MultiagentSystems/README.md) | | |
|
||||
| VII | **人工智能倫理** | | |
|
||||
| 24 | [人工智能倫理與負責任的 AI](./lessons/7-Ethics/README.md) | [Microsoft Learn：負責任的 AI 原則](https://docs.microsoft.com/learn/paths/responsible-ai-business-principles/?WT.mc_id=academic-77998-cacaste) | |
|
||||
| IX | **附加內容** | | |
|
||||
| 25 | [多模態網絡、CLIP 與 VQGAN](./lessons/X-Extras/X1-MultiModal/README.md) | [筆記本](./lessons/X-Extras/X1-MultiModal/Clip.ipynb) | |
|
||||
|
||||
## 每課包含
|
||||
|
||||
* 預習資料
|
||||
* 可執行的 Jupyter 筆記本，通常針對特定框架（**PyTorch** 或 **TensorFlow**）。這些可執行筆記本也包含大量理論內容，因此要理解主題，你至少需要完成其中一種框架的筆記本（PyTorch 或 TensorFlow）。
|
||||
* 部分主題提供 **實驗室**，讓你有機會將所學運用於特定問題。
|
||||
* 部分章節包含指向涵蓋相關主題的 [**MS Learn**](https://learn.microsoft.com/en-us/collections/7w28iy2xrqzdj0?WT.mc_id=academic-77998-bethanycheum) 模組連結。
|
||||
|
||||
## 入門指南
|
||||
|
||||
### 🎯 AI 新手？從這裡開始！
|
||||
|
||||
如果你是 AI 完全新手，想要快速、動手的範例，請查看我們的 [**新手友好範例**](./examples/README.md)！這些包括：
|
||||
|
||||
- 🌟 **Hello AI World** — 你的第一個 AI 程式（模式識別）
|
||||
- 🧠 **簡單神經網絡** — 從零開始構建神經網絡
|
||||
|
||||
- 🖼️ **影像分類器** - 以詳細註解分類影像
|
||||
- 💬 **文字情感分析** - 分析正面/負面文字
|
||||
|
||||
這些範例旨在幫助你在深入完整課程前理解 AI 概念。
|
||||
|
||||
### 📚 完整課程設定
|
||||
|
||||
- 我們已建立一個[設定課程](./lessons/0-course-setup/setup.md)協助你設定開發環境。 - 對教育工作者，我們也製作了一個[課程設定課程](./lessons/0-course-setup/for-teachers.md)！
|
||||
- 如何在 VSCode 或 Codespace 中[運行程式碼](./lessons/0-course-setup/how-to-run.md)
|
||||
|
||||
請依照以下步驟：
|
||||
|
||||
派生此儲存庫：點擊本頁右上角的「Fork」按鈕。
|
||||
|
||||
克隆此儲存庫：`git clone https://github.com/microsoft/AI-For-Beginners.git`
|
||||
|
||||
別忘了收藏（🌟）此儲存庫，以便日後更容易找到。
|
||||
|
||||
## 與其他學習者見面
|
||||
|
||||
加入我們的[官方 AI Discord 伺服器](https://aka.ms/genai-discord?WT.mc_id=academic-105485-bethanycheum)，與正在修讀此課程的學習者交流並獲得支援。
|
||||
|
||||
如果你在製作過程中有產品反饋或問題，請造訪我們的[Azure AI Foundry 開發者論壇](https://aka.ms/foundry/forum)
|
||||
|
||||
## 測驗
|
||||
|
||||
> **關於測驗的說明**：所有測驗均存放於 etc\quiz-app 的 Quiz-app 資料夾，或可在[此處線上使用](https://ff-quizzes.netlify.app/)。測驗由課程內頁連結調用，測驗應用程式可在本機端執行或部署到 Azure；請遵循`quiz-app`資料夾內的說明。測驗正在逐步本地化。
|
||||
|
||||
## 求助
|
||||
|
||||
你有建議或發現拼寫或程式碼錯誤嗎？請提出問題單或建立拉取請求。
|
||||
|
||||
## 特別感謝
|
||||
|
||||
* **✍️ 主要作者:** [Dmitry Soshnikov](http://soshnikov.com), 博士
|
||||
* **🔥 編輯:** [Jen Looper](https://twitter.com/jenlooper), 博士
|
||||
* **🎨 速寫插畫:** [Tomomi Imura](https://twitter.com/girlie_mac)
|
||||
* **✅ 測驗創作者:** [Lateefah Bello](https://github.com/CinnamonXI), [MLSA](https://studentambassadors.microsoft.com/)
|
||||
* **🙏 核心貢獻者:** [Evgenii Pishchik](https://github.com/Pe4enIks)
|
||||
|
||||
## 其他課程
|
||||
|
||||
我們團隊還製作了其他課程！歡迎查看：
|
||||
|
||||
<!-- CO-OP TRANSLATOR OTHER COURSES START -->
|
||||
### LangChain
|
||||
[](https://aka.ms/langchain4j-for-beginners)
|
||||
[](https://aka.ms/langchainjs-for-beginners?WT.mc_id=m365-94501-dwahlin)
|
||||
|
||||
---
|
||||
|
||||
### Azure / Edge / MCP / Agents
|
||||
[](https://github.com/microsoft/AZD-for-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://github.com/microsoft/edgeai-for-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://github.com/microsoft/mcp-for-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://github.com/microsoft/ai-agents-for-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
|
||||
---
|
||||
|
||||
### 生成式 AI 系列
|
||||
[](https://github.com/microsoft/generative-ai-for-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[-9333EA?style=for-the-badge&labelColor=E5E7EB&color=9333EA)](https://github.com/microsoft/Generative-AI-for-beginners-dotnet?WT.mc_id=academic-105485-koreyst)
|
||||
[-C084FC?style=for-the-badge&labelColor=E5E7EB&color=C084FC)](https://github.com/microsoft/generative-ai-for-beginners-java?WT.mc_id=academic-105485-koreyst)
|
||||
[-E879F9?style=for-the-badge&labelColor=E5E7EB&color=E879F9)](https://github.com/microsoft/generative-ai-with-javascript?WT.mc_id=academic-105485-koreyst)
|
||||
|
||||
---
|
||||
|
||||
### 核心學習
|
||||
[](https://aka.ms/ml-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://aka.ms/datascience-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://aka.ms/ai-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://github.com/microsoft/Security-101?WT.mc_id=academic-96948-sayoung)
|
||||
[](https://aka.ms/webdev-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://aka.ms/iot-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://github.com/microsoft/xr-development-for-beginners?WT.mc_id=academic-105485-koreyst)
|
||||
|
||||
---
|
||||
|
||||
### Copilot 系列
|
||||
[](https://aka.ms/GitHubCopilotAI?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://github.com/microsoft/mastering-github-copilot-for-dotnet-csharp-developers?WT.mc_id=academic-105485-koreyst)
|
||||
[](https://github.com/microsoft/CopilotAdventures?WT.mc_id=academic-105485-koreyst)
|
||||
<!-- CO-OP TRANSLATOR OTHER COURSES END -->
|
||||
|
||||
## 尋求協助
|
||||
|
||||
如果你在開發 AI 應用時遇到困難或有任何疑問，歡迎加入學習者和經驗豐富開發者的討論區，一同探討 MCP。這是一個支持性強的社群，歡迎提問並自由分享知識。
|
||||
|
||||
[](https://discord.gg/nTYy5BXMWG)
|
||||
|
||||
若你在開發過程中有產品反饋或遇到錯誤，歡迎造訪：
|
||||
|
||||
[](https://aka.ms/foundry/forum)
|
||||
|
||||
---
|
||||
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER START -->
|
||||
**免責聲明**：
|
||||
本文件是使用人工智能翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們力求準確，但請注意自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於重要信息，建議使用專業人工翻譯。因使用本翻譯所引起的任何誤解或誤釋，我們概不負責。
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER END -->
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
## 安全性
|
||||
|
||||
Microsoft 非常重視我們軟體產品與服務的安全性，這包括透過我們的 GitHub 組織管理的所有原始碼庫，這些組織包括 [Microsoft](https://github.com/Microsoft)、[Azure](https://github.com/Azure)、[DotNet](https://github.com/dotnet)、[AspNet](https://github.com/aspnet)、[Xamarin](https://github.com/xamarin) 和 [我們的 GitHub 組織](https://opensource.microsoft.com/)。
|
||||
|
||||
如果您認為在任何 Microsoft 擁有的原始碼庫中發現了符合 [Microsoft 對安全性漏洞的定義](https://aka.ms/opensource/security/definition) 的安全性漏洞，請按照以下說明向我們報告。
|
||||
|
||||
## 回報安全性問題
|
||||
|
||||
**請勿透過公開的 GitHub 問題回報安全性漏洞。**
|
||||
|
||||
相反地，請透過 Microsoft Security Response Center (MSRC) 回報，網址為 [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report)。
|
||||
|
||||
如果您希望在不登入的情況下提交，請發送電子郵件至 [secure@microsoft.com](mailto:secure@microsoft.com)。如果可能，請使用我們的 PGP 金鑰加密您的訊息；您可以從 [Microsoft Security Response Center PGP Key 頁面](https://aka.ms/opensource/security/pgpkey) 下載。
|
||||
|
||||
您應該會在 24 小時內收到回應。如果因某些原因未收到回應，請透過電子郵件跟進，以確保我們已收到您的原始訊息。更多資訊請參閱 [microsoft.com/msrc](https://aka.ms/opensource/security/msrc)。
|
||||
|
||||
請盡可能提供以下所列的資訊，以幫助我們更好地理解問題的性質與範圍：
|
||||
|
||||
* 問題類型（例如：緩衝區溢位、SQL 注入、跨網站腳本攻擊等）
|
||||
* 與問題相關的原始檔案完整路徑
|
||||
* 受影響原始碼的位置（標籤/分支/提交或直接 URL）
|
||||
* 重現問題所需的任何特殊配置
|
||||
* 重現問題的逐步指導
|
||||
* 概念驗證或利用程式碼（如果可能）
|
||||
* 問題的影響，包括攻擊者可能如何利用該問題
|
||||
|
||||
這些資訊將幫助我們更快速地處理您的回報。
|
||||
|
||||
如果您是為了漏洞賞金計畫而回報，提供更完整的報告可能有助於獲得更高的賞金獎勵。請造訪我們的 [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) 頁面，了解更多關於我們現行計畫的詳情。
|
||||
|
||||
## 偏好語言
|
||||
|
||||
我們偏好所有的溝通以英文進行。
|
||||
|
||||
## 政策
|
||||
|
||||
Microsoft 遵循 [協調式漏洞揭露原則](https://aka.ms/opensource/security/cvd)。
|
||||
|
||||
**免責聲明**：
|
||||
本文檔已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。我們致力於提供準確的翻譯，但請注意，自動翻譯可能包含錯誤或不準確之處。應以原始語言的文件作為權威來源。對於關鍵信息，建議尋求專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或誤釋不承擔責任。
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
# Microsoft 開源行為準則
|
||||
|
||||
此專案已採用 [Microsoft 開源行為準則](https://opensource.microsoft.com/codeofconduct/)。
|
||||
|
||||
資源：
|
||||
|
||||
- [Microsoft 開源行為準則](https://opensource.microsoft.com/codeofconduct/)
|
||||
- [Microsoft 行為準則常見問題](https://opensource.microsoft.com/codeofconduct/faq/)
|
||||
- 如有任何問題或疑慮，請聯絡 [opencode@microsoft.com](mailto:opencode@microsoft.com)
|
||||
|
||||
**免責聲明**：
|
||||
本文件使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。我們致力於提供準確的翻譯，但請注意，自動翻譯可能包含錯誤或不準確之處。應以原文文件作為權威來源。對於關鍵資訊，建議尋求專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或誤讀概不負責。
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
# 貢獻指南
|
||||
|
||||
此專案歡迎各種貢獻與建議。大多數的貢獻需要您同意一份貢獻者授權協議 (CLA)，以聲明您擁有權利並實際授予我們使用您的貢獻的權利。詳細資訊請參閱 https://cla.microsoft.com。
|
||||
|
||||
當您提交拉取請求 (pull request) 時，CLA 機器人會自動判斷您是否需要提供 CLA，並適當地標記 PR（例如，添加標籤或評論）。只需按照機器人提供的指示操作即可。您只需在所有使用我們 CLA 的存儲庫中完成一次此操作。
|
||||
|
||||
此專案已採用 [Microsoft 開源行為準則](https://opensource.microsoft.com/codeofconduct/)。如需更多資訊，請參閱 [行為準則常見問題](https://opensource.microsoft.com/codeofconduct/faq/) 或聯絡 [opencode@microsoft.com](mailto:opencode@microsoft.com) 提出其他問題或意見。
|
||||
|
||||
# 尋求貢獻
|
||||
|
||||
我們目前正在積極尋求以下主題的貢獻：
|
||||
|
||||
- [ ] 撰寫深度強化學習的章節
|
||||
- [ ] 改善物件偵測的章節與筆記本
|
||||
- [ ] PyTorch Lightning（針對[此章節](https://github.com/microsoft/AI-For-Beginners/blob/main/3-NeuralNetworks/05-Frameworks/README.md)）
|
||||
- [ ] 撰寫命名實體識別的章節與範例
|
||||
- [ ] 為[此章節](https://github.com/microsoft/AI-For-Beginners/tree/main/5-NLP/15-LanguageModeling)建立訓練自有嵌入的範例
|
||||
|
||||
**免責聲明**：
|
||||
本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們努力確保翻譯的準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解釋不承擔責任。
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
# 人工智慧 (AI)
|
||||
|
||||
## [人工智慧簡介](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/1-Intro/README.md)
|
||||
- 人工智慧的定義
|
||||
- 人工智慧的歷史
|
||||
- 人工智慧的方法
|
||||
- 自上而下/符號式
|
||||
- 自下而上/神經網路
|
||||
- 演化式
|
||||
- 協同/湧現式人工智慧
|
||||
- [微軟人工智慧商業學院](https://www.microsoft.com/ai/ai-business-school/?WT.mc_id=academic-77998-cacaste)
|
||||
|
||||
## [符號式人工智慧](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/2-Symbolic/README.md)
|
||||
- 知識表示
|
||||
- [專家系統](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/2-Symbolic/Animals.ipynb)
|
||||
- [本體論](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/2-Symbolic/FamilyOntology.ipynb)
|
||||
- 語意網
|
||||
|
||||
## [神經網路](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/README.md)
|
||||
- [感知器](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/03-Perceptron/README.md)
|
||||
- [多層網路](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/04-OwnFramework/README.md)
|
||||
- [框架簡介](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/05-Frameworks/README.md)
|
||||
- [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb)
|
||||
- [TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/05-Frameworks/IntroKerasTF.md)
|
||||
- [過擬合](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/05-Frameworks/Overfitting.md)
|
||||
|
||||
## [電腦視覺](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/README.md)
|
||||
- 在微軟學習平台上
|
||||
- [人工智慧基礎：探索電腦視覺](https://docs.microsoft.com/learn/paths/explore-computer-vision-microsoft-azure/?WT.mc_id=academic-77998-cacaste)
|
||||
- [使用 PyTorch 的電腦視覺](https://docs.microsoft.com/learn/modules/intro-computer-vision-pytorch/?WT.mc_id=academic-77998-cacaste)
|
||||
- [使用 TensorFlow 的電腦視覺](https://docs.microsoft.com/learn/modules/intro-computer-vision-TensorFlow/?WT.mc_id=academic-77998-cacaste)
|
||||
- [電腦視覺簡介：OpenCV](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/06-IntroCV/README.md)
|
||||
- [卷積網路](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/07-ConvNets/README.md)
|
||||
- [卷積神經網路架構](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/07-ConvNets/CNN_Architectures.md)
|
||||
- [遷移學習](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/08-TransferLearning/README.md)
|
||||
- [訓練技巧](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/08-TransferLearning/TrainingTricks.md)
|
||||
- [自編碼器與變分自編碼器 (VAEs)](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/09-Autoencoders/README.md)
|
||||
- [生成對抗網路 (GANs)](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/README.md)
|
||||
- [風格遷移](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/StyleTransfer.ipynb)
|
||||
- [物件偵測](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/11-ObjectDetection/README.md)
|
||||
- [影像分割](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/12-Segmentation/README.md)
|
||||
|
||||
## [自然語言處理](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/README.md)
|
||||
- 在微軟學習平台上
|
||||
- [人工智慧基礎：探索自然語言處理](https://docs.microsoft.com/learn/paths/explore-natural-language-processing/?WT.mc_id=academic-77998-cacaste)
|
||||
- [使用 PyTorch 的自然語言處理](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-pytorch/?WT.mc_id=academic-77998-cacaste)
|
||||
- [使用 TensorFlow 的自然語言處理](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-TensorFlow/?WT.mc_id=academic-77998-cacaste)
|
||||
- [文本表示](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/13-TextRep/README.md)
|
||||
- 詞袋模型 (Bag of Words)
|
||||
- TF/IDF
|
||||
- [語意嵌入](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/14-Embeddings/README.md)
|
||||
- Word2Vec
|
||||
- GloVE
|
||||
- [語言建模](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/15-LanguageModeling)
|
||||
- [循環神經網路 (RNN)](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/16-RNN/README.md)
|
||||
- 長短期記憶網路 (LSTM)
|
||||
- 閘控循環單元 (GRU)
|
||||
- [生成式循環網路](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/17-GenerativeNetworks/README.md)
|
||||
- [Transformer 和 BERT](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/18-Transformers/README.md)
|
||||
- [命名實體識別 (NER)](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/19-NER/README.md)
|
||||
- [文本生成與 GPT](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/20-LanguageModels/README.md)
|
||||
|
||||
## 其他技術
|
||||
- [遺傳演算法](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/6-Other/21-GeneticAlgorithms/README.md)
|
||||
- [深度強化學習](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/6-Other/22-DeepRL/README.md)
|
||||
- [多代理系統](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/6-Other/23-MultiagentSystems/README.md)
|
||||
|
||||
## [人工智慧倫理](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/7-Ethics/README.md)
|
||||
- [微軟學習平台上的負責任人工智慧](https://docs.microsoft.com/learn/paths/responsible-ai-business-principles/?WT.mc_id=academic-77998-cacaste)
|
||||
|
||||
## 附加內容
|
||||
- [多模態網路](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/X-Extras/X1-MultiModal/README.md)
|
||||
- [CLIP](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/X-Extras/X1-MultiModal/Clip.ipynb)
|
||||
- DALL-E
|
||||
- VQ-GAN
|
||||
|
||||
**免責聲明**：
|
||||
本文檔使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們努力確保準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始語言的文件應被視為權威來源。對於關鍵信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或誤釋不承擔責任。
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# 支援
|
||||
|
||||
## 如何提交問題並獲得幫助
|
||||
|
||||
此專案使用 GitHub Issues 來追蹤錯誤和功能請求。在提交新問題之前，請先搜尋現有的問題以避免重複。對於新問題，請將您的錯誤或功能請求提交為一個新的 Issue。
|
||||
|
||||
若需幫助或對使用此專案有任何疑問，請使用討論板。
|
||||
|
||||
## Microsoft 支援政策
|
||||
|
||||
此專案的支援僅限於上述列出的資源。
|
||||
|
||||
**免責聲明**：
|
||||
本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們努力確保翻譯的準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或誤釋不承擔責任。
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
# 透過翻譯課程內容來貢獻
|
||||
|
||||
我們歡迎您為本課程的內容進行翻譯！
|
||||
|
||||
## 指導方針
|
||||
|
||||
在每個課程資料夾和課程介紹資料夾中，都有包含翻譯後的 Markdown 文件的資料夾。
|
||||
|
||||
> 注意，請勿翻譯任何程式碼範例文件中的程式碼；唯一需要翻譯的部分是 README、作業和測驗。謝謝！
|
||||
|
||||
翻譯後的文件應遵循以下命名規則：
|
||||
|
||||
**README._[language]_.md**
|
||||
|
||||
其中 _[language]_ 是遵循 ISO 639-1 標準的兩位字母語言縮寫（例如，西班牙文為 `README.es.md`，荷蘭文為 `README.nl.md`）。
|
||||
|
||||
**assignment._[language]_.md**
|
||||
|
||||
與 README 文件類似，請翻譯作業文件。
|
||||
|
||||
**測驗**
|
||||
|
||||
1. 將您的翻譯添加到測驗應用程式中，方法是將文件添加到以下位置：https://github.com/microsoft/AI-For-Beginners/tree/main/etc/quiz-app/src/assets/translations，並遵循正確的命名規則（例如，en.json、fr.json）。**請不要翻譯 "true" 或 "false" 這些詞語，謝謝！**
|
||||
|
||||
2. 將您的語言代碼添加到測驗應用程式的 App.vue 文件中的下拉選單。
|
||||
|
||||
3. 編輯測驗應用程式的 [translations index.js 文件](https://github.com/microsoft/AI-For-Beginners/blob/main/etc/quiz-app/src/assets/translations/index.js)，以添加您的語言。
|
||||
|
||||
4. 最後，編輯您翻譯後的 README.md 文件中的所有測驗連結，直接指向您的翻譯後測驗：https://red-field-0a6ddfd03.1.azurestaticapps.net/quiz/1 改為 https://red-field-0a6ddfd03.1.azurestaticapps.net/quiz/1?loc=id
|
||||
|
||||
**感謝您**
|
||||
|
||||
我們真心感謝您的努力！
|
||||
|
||||
**免責聲明**：
|
||||
本文件使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。雖然我們努力確保翻譯的準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解釋不承擔責任。
|
||||
|
|
@ -0,0 +1,128 @@
|
|||
# 測驗
|
||||
|
||||
這些測驗是 AI 課程的課前和課後測驗，課程網址為：https://aka.ms/ai-beginners
|
||||
|
||||
## 新增翻譯的測驗集
|
||||
|
||||
要新增測驗翻譯，請在 `assets/translations` 資料夾中建立對應的測驗結構。原始的測驗位於 `assets/translations/en`。測驗依據課程分為多個部分，請確保編號與正確的測驗部分對齊。整個課程共有 40 個測驗，編號從 0 開始。
|
||||
|
||||
完成翻譯後，請編輯翻譯資料夾中的 `index.js` 檔案，按照 `en` 資料夾中的慣例匯入所有檔案。
|
||||
|
||||
接著，編輯 `assets/translations` 資料夾中的 `index.js` 檔案，匯入新的翻譯檔案。
|
||||
|
||||
然後，編輯此應用程式中的 `App.vue` 檔案中的下拉選單，新增您的語言。請確保本地化的縮寫與您的語言資料夾名稱一致。
|
||||
|
||||
最後，如果翻譯課程中已存在測驗連結，請編輯所有測驗連結，將本地化參數加入查詢字串中，例如：`?loc=fr`。
|
||||
|
||||
## 專案設定
|
||||
|
||||
```
|
||||
npm install
|
||||
```
|
||||
|
||||
### 編譯並啟用開發模式的即時重新載入
|
||||
|
||||
```
|
||||
npm run serve
|
||||
```
|
||||
|
||||
### 編譯並壓縮以進行生產部署
|
||||
|
||||
```
|
||||
npm run build
|
||||
```
|
||||
|
||||
### 檢查並修復檔案
|
||||
|
||||
```
|
||||
npm run lint
|
||||
```
|
||||
|
||||
### 自訂設定
|
||||
|
||||
請參閱 [設定參考](https://cli.vuejs.org/config/)。
|
||||
|
||||
致謝：感謝此測驗應用程式的原始版本：https://github.com/arpan45/simple-quiz-vue
|
||||
|
||||
## 部署到 Azure
|
||||
|
||||
以下是幫助您開始的逐步指南：
|
||||
|
||||
1. **Fork GitHub 儲存庫**
|
||||
確保您的靜態網站應用程式程式碼位於您的 GitHub 儲存庫中。Fork 此儲存庫。
|
||||
|
||||
2. **建立 Azure 靜態網站應用程式**
|
||||
- 建立並登入 [Azure 帳戶](http://azure.microsoft.com)
|
||||
- 前往 [Azure 入口網站](https://portal.azure.com)
|
||||
- 點擊「建立資源」，搜尋「靜態網站應用程式」。
|
||||
- 點擊「建立」。
|
||||
|
||||
3. **設定靜態網站應用程式**
|
||||
- **基本設定**：
|
||||
- 訂閱：選擇您的 Azure 訂閱。
|
||||
- 資源群組：建立新的資源群組或使用現有的。
|
||||
- 名稱：為您的靜態網站應用程式提供一個名稱。
|
||||
- 區域：選擇最接近使用者的區域。
|
||||
|
||||
- **部署詳細資訊**：
|
||||
- 原始碼：選擇「GitHub」。
|
||||
- GitHub 帳戶：授權 Azure 存取您的 GitHub 帳戶。
|
||||
- 組織：選擇您的 GitHub 組織。
|
||||
- 儲存庫：選擇包含靜態網站應用程式的儲存庫。
|
||||
- 分支：選擇您要部署的分支。
|
||||
|
||||
- **建置詳細資訊**：
|
||||
- 建置預設值：選擇應用程式使用的框架（例如 React、Angular、Vue 等）。
|
||||
- 應用程式位置：指定應用程式程式碼所在的資料夾（例如，如果在根目錄，則為 `/`）。
|
||||
- API 位置：如果有 API，請指定其位置（可選）。
|
||||
- 輸出位置：指定建置輸出生成的資料夾（例如 build 或 dist）。
|
||||
|
||||
4. **檢查並建立**
|
||||
檢查您的設定，然後點擊「建立」。Azure 會設置必要的資源，並在您的儲存庫中建立 GitHub Actions 工作流程。
|
||||
|
||||
5. **GitHub Actions 工作流程**
|
||||
Azure 會自動在您的儲存庫中建立 GitHub Actions 工作流程檔案（.github/workflows/azure-static-web-apps-<name>.yml）。此工作流程將處理建置和部署過程。
|
||||
|
||||
6. **監控部署**
|
||||
前往 GitHub 儲存庫中的「Actions」標籤。
|
||||
您應該會看到一個正在運行的工作流程。此工作流程將建置並部署您的靜態網站應用程式到 Azure。
|
||||
工作流程完成後，您的應用程式將在提供的 Azure URL 上線。
|
||||
|
||||
### 範例工作流程檔案
|
||||
|
||||
以下是 GitHub Actions 工作流程檔案的範例：
|
||||
name: Azure Static Web Apps CI/CD
|
||||
```
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened, closed]
|
||||
branches:
|
||||
- main
|
||||
|
||||
jobs:
|
||||
build_and_deploy_job:
|
||||
runs-on: ubuntu-latest
|
||||
name: Build and Deploy Job
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- name: Build And Deploy
|
||||
id: builddeploy
|
||||
uses: Azure/static-web-apps-deploy@v1
|
||||
with:
|
||||
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN }}
|
||||
repo_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
action: "upload"
|
||||
app_location: "etc/quiz-app # App source code path"
|
||||
api_location: ""API source code path optional
|
||||
output_location: "dist" #Built app content directory - optional
|
||||
```
|
||||
|
||||
### 其他資源
|
||||
- [Azure 靜態網站應用程式文件](https://learn.microsoft.com/azure/static-web-apps/getting-started)
|
||||
- [GitHub Actions 文件](https://docs.github.com/actions/use-cases-and-examples/deploying/deploying-to-azure-static-web-app)
|
||||
|
||||
**免責聲明**：
|
||||
本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。雖然我們努力確保翻譯的準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解釋不承擔責任。
|
||||
|
|
@ -0,0 +1,397 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 簡單圖像分類器\n",
|
||||
"\n",
|
||||
"此筆記本將向您展示如何使用預訓練的神經網絡進行圖像分類。\n",
|
||||
"\n",
|
||||
"**您將學到：**\n",
|
||||
"- 如何載入並使用預訓練模型\n",
|
||||
"- 圖像預處理\n",
|
||||
"- 對圖像進行預測\n",
|
||||
"- 理解信心分數\n",
|
||||
"\n",
|
||||
"**使用案例：** 識別圖像中的物體（例如「貓」、「狗」、「車」等）\n",
|
||||
"\n",
|
||||
"---\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 第一步：匯入所需的庫\n",
|
||||
"\n",
|
||||
"讓我們匯入所需的工具。不用擔心，如果你現在還不完全理解這些！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Core libraries\n",
|
||||
"import numpy as np\n",
|
||||
"from PIL import Image\n",
|
||||
"import requests\n",
|
||||
"from io import BytesIO\n",
|
||||
"\n",
|
||||
"# TensorFlow for deep learning\n",
|
||||
"try:\n",
|
||||
" import tensorflow as tf\n",
|
||||
" from tensorflow.keras.applications import MobileNetV2\n",
|
||||
" from tensorflow.keras.applications.mobilenet_v2 import preprocess_input, decode_predictions\n",
|
||||
" print(\"✅ TensorFlow loaded successfully!\")\n",
|
||||
" print(f\" Version: {tf.__version__}\")\n",
|
||||
"except ImportError:\n",
|
||||
" print(\"❌ Please install TensorFlow: pip install tensorflow\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 步驟 2：載入預訓練模型\n",
|
||||
"\n",
|
||||
"我們將使用 **MobileNetV2**，這是一個已經在數百萬張圖片上訓練過的神經網絡。\n",
|
||||
"\n",
|
||||
"這被稱為 **遷移學習**——使用別人訓練好的模型！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(\"📦 Loading pre-trained MobileNetV2 model...\")\n",
|
||||
"print(\" This may take a minute on first run (downloading weights)...\")\n",
|
||||
"\n",
|
||||
"# Load the model\n",
|
||||
"# include_top=True means we use the classification layer\n",
|
||||
"# weights='imagenet' means it was trained on ImageNet dataset\n",
|
||||
"model = MobileNetV2(weights='imagenet', include_top=True)\n",
|
||||
"\n",
|
||||
"print(\"✅ Model loaded!\")\n",
|
||||
"print(f\" The model can recognize 1000 different object categories\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 第 3 步：輔助函式\n",
|
||||
"\n",
|
||||
"讓我們建立一些函式來載入並準備影像，以供我們的模型使用。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def load_image_from_url(url):\n",
|
||||
" \"\"\"\n",
|
||||
" Load an image from a URL.\n",
|
||||
" \n",
|
||||
" Args:\n",
|
||||
" url: Web address of the image\n",
|
||||
" \n",
|
||||
" Returns:\n",
|
||||
" PIL Image object\n",
|
||||
" \"\"\"\n",
|
||||
" response = requests.get(url)\n",
|
||||
" img = Image.open(BytesIO(response.content))\n",
|
||||
" return img\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def prepare_image(img):\n",
|
||||
" \"\"\"\n",
|
||||
" Prepare an image for the model.\n",
|
||||
" \n",
|
||||
" Steps:\n",
|
||||
" 1. Resize to 224x224 (model's expected size)\n",
|
||||
" 2. Convert to array\n",
|
||||
" 3. Add batch dimension\n",
|
||||
" 4. Preprocess for MobileNetV2\n",
|
||||
" \n",
|
||||
" Args:\n",
|
||||
" img: PIL Image\n",
|
||||
" \n",
|
||||
" Returns:\n",
|
||||
" Preprocessed image array\n",
|
||||
" \"\"\"\n",
|
||||
" # Resize to 224x224 pixels\n",
|
||||
" img = img.resize((224, 224))\n",
|
||||
" \n",
|
||||
" # Convert to numpy array\n",
|
||||
" img_array = np.array(img)\n",
|
||||
" \n",
|
||||
" # Add batch dimension (model expects multiple images)\n",
|
||||
" img_array = np.expand_dims(img_array, axis=0)\n",
|
||||
" \n",
|
||||
" # Preprocess for MobileNetV2\n",
|
||||
" img_array = preprocess_input(img_array)\n",
|
||||
" \n",
|
||||
" return img_array\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def classify_image(img):\n",
|
||||
" \"\"\"\n",
|
||||
" Classify an image and return top predictions.\n",
|
||||
" \n",
|
||||
" Args:\n",
|
||||
" img: PIL Image\n",
|
||||
" \n",
|
||||
" Returns:\n",
|
||||
" List of (class_name, confidence) tuples\n",
|
||||
" \"\"\"\n",
|
||||
" # Prepare the image\n",
|
||||
" img_array = prepare_image(img)\n",
|
||||
" \n",
|
||||
" # Make prediction\n",
|
||||
" predictions = model.predict(img_array, verbose=0)\n",
|
||||
" \n",
|
||||
" # Decode predictions to human-readable labels\n",
|
||||
" # top=5 means we get the top 5 most likely classes\n",
|
||||
" decoded = decode_predictions(predictions, top=5)[0]\n",
|
||||
" \n",
|
||||
" # Convert to simpler format\n",
|
||||
" results = [(label, float(confidence)) for (_, label, confidence) in decoded]\n",
|
||||
" \n",
|
||||
" return results\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"print(\"✅ Helper functions ready!\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 第四步：使用範例圖片進行測試\n",
|
||||
"\n",
|
||||
"讓我們嘗試分類一些來自網路的圖片吧！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Sample images to classify\n",
|
||||
"# These are from Unsplash (free stock photos)\n",
|
||||
"test_images = [\n",
|
||||
" {\n",
|
||||
" \"url\": \"https://images.unsplash.com/photo-1514888286974-6c03e2ca1dba?w=400\",\n",
|
||||
" \"description\": \"A cat\"\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"url\": \"https://images.unsplash.com/photo-1552053831-71594a27632d?w=400\",\n",
|
||||
" \"description\": \"A dog\"\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"url\": \"https://images.unsplash.com/photo-1511919884226-fd3cad34687c?w=400\",\n",
|
||||
" \"description\": \"A car\"\n",
|
||||
" },\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"print(f\"🧪 Testing on {len(test_images)} images...\")\n",
|
||||
"print(\"=\" * 70)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 將每張圖片分類\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for i, img_data in enumerate(test_images, 1):\n",
|
||||
" print(f\"\\n📸 Image {i}: {img_data['description']}\")\n",
|
||||
" print(\"-\" * 70)\n",
|
||||
" \n",
|
||||
" try:\n",
|
||||
" # Load image\n",
|
||||
" img = load_image_from_url(img_data['url'])\n",
|
||||
" \n",
|
||||
" # Display image\n",
|
||||
" display(img.resize((200, 200))) # Show smaller version\n",
|
||||
" \n",
|
||||
" # Classify\n",
|
||||
" results = classify_image(img)\n",
|
||||
" \n",
|
||||
" # Show predictions\n",
|
||||
" print(\"\\n🎯 Top 5 Predictions:\")\n",
|
||||
" for rank, (label, confidence) in enumerate(results, 1):\n",
|
||||
" # Create a visual bar\n",
|
||||
" bar_length = int(confidence * 50)\n",
|
||||
" bar = \"█\" * bar_length\n",
|
||||
" \n",
|
||||
" print(f\" {rank}. {label:20s} {confidence*100:5.2f}% {bar}\")\n",
|
||||
" \n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"❌ Error: {e}\")\n",
|
||||
"\n",
|
||||
"print(\"\\n\" + \"=\" * 70)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 第五步：嘗試使用自己的圖片！\n",
|
||||
"\n",
|
||||
"將下面的 URL 替換為您想要分類的任何圖片 URL。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Try your own image!\n",
|
||||
"# Replace this URL with any image URL\n",
|
||||
"custom_image_url = \"https://images.unsplash.com/photo-1472491235688-bdc81a63246e?w=400\" # A flower\n",
|
||||
"\n",
|
||||
"print(\"🖼️ Classifying your custom image...\")\n",
|
||||
"print(\"=\" * 70)\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" # Load and show image\n",
|
||||
" img = load_image_from_url(custom_image_url)\n",
|
||||
" display(img.resize((300, 300)))\n",
|
||||
" \n",
|
||||
" # Classify\n",
|
||||
" results = classify_image(img)\n",
|
||||
" \n",
|
||||
" # Show results\n",
|
||||
" print(\"\\n🎯 Top 5 Predictions:\")\n",
|
||||
" print(\"-\" * 70)\n",
|
||||
" for rank, (label, confidence) in enumerate(results, 1):\n",
|
||||
" bar_length = int(confidence * 50)\n",
|
||||
" bar = \"█\" * bar_length\n",
|
||||
" print(f\" {rank}. {label:20s} {confidence*100:5.2f}% {bar}\")\n",
|
||||
" \n",
|
||||
" # Highlight top prediction\n",
|
||||
" top_label, top_confidence = results[0]\n",
|
||||
" print(\"\\n\" + \"=\" * 70)\n",
|
||||
" print(f\"\\n🏆 Best guess: {top_label} ({top_confidence*100:.2f}% confident)\")\n",
|
||||
" \n",
|
||||
"except Exception as e:\n",
|
||||
" print(f\"❌ Error: {e}\")\n",
|
||||
" print(\" Make sure the URL points to a valid image!\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 💡 剛剛發生了什麼？\n",
|
||||
"\n",
|
||||
"1. **我們載入了一個預訓練模型** - MobileNetV2 已經在數百萬張圖片上進行過訓練 \n",
|
||||
"2. **我們對圖片進行了預處理** - 將圖片調整大小並格式化以適配模型 \n",
|
||||
"3. **模型進行了預測** - 輸出了對 1000 個物件類別的概率 \n",
|
||||
"4. **我們解碼了結果** - 將數字轉換為人類可讀的標籤 \n",
|
||||
"\n",
|
||||
"### 理解信心分數\n",
|
||||
"\n",
|
||||
"- **90-100%**：非常有信心（幾乎肯定正確） \n",
|
||||
"- **70-90%**：有信心（可能正確） \n",
|
||||
"- **50-70%**：信心一般（可能正確） \n",
|
||||
"- **低於 50%**：信心不足（不確定） \n",
|
||||
"\n",
|
||||
"### 為什麼預測可能會出錯？\n",
|
||||
"\n",
|
||||
"- **不尋常的角度或光線** - 模型是在典型照片上訓練的 \n",
|
||||
"- **多個物件** - 模型預期只有一個主要物件 \n",
|
||||
"- **罕見物件** - 模型只認識 1000 個類別 \n",
|
||||
"- **低品質圖片** - 模糊或像素化的圖片更難辨識 \n",
|
||||
"\n",
|
||||
"---\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 🚀 下一步\n",
|
||||
"\n",
|
||||
"1. **嘗試不同的圖片：**\n",
|
||||
" - 在 [Unsplash](https://unsplash.com) 找圖片\n",
|
||||
" - 右鍵 → 「複製圖片地址」以獲取 URL\n",
|
||||
"\n",
|
||||
"2. **試驗：**\n",
|
||||
" - 抽象藝術會有什麼效果？\n",
|
||||
" - 它能否辨識不同角度的物體？\n",
|
||||
" - 它如何處理多個物體？\n",
|
||||
"\n",
|
||||
"3. **深入了解：**\n",
|
||||
" - 探索 [電腦視覺課程](../lessons/4-ComputerVision/README.md)\n",
|
||||
" - 學習如何訓練自己的圖片分類器\n",
|
||||
" - 理解 CNN（卷積神經網絡）的工作原理\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"## 🎉 恭喜！\n",
|
||||
"\n",
|
||||
"你剛剛使用最先進的神經網絡建造了一個圖片分類器！\n",
|
||||
"\n",
|
||||
"這項技術同樣應用於：\n",
|
||||
"- Google Photos（整理你的照片）\n",
|
||||
"- 自駕車（辨識物體）\n",
|
||||
"- 醫療診斷（分析 X 光片）\n",
|
||||
"- 品質控制（檢測缺陷）\n",
|
||||
"\n",
|
||||
"繼續探索和學習吧！🚀\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免責聲明**： \n本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們努力確保翻譯的準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵資訊，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解釋不承擔責任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.0"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "1d472141d9df46b751542b3c29f88677",
|
||||
"translation_date": "2025-10-03T11:39:54+00:00",
|
||||
"source_file": "examples/03-image-classifier.ipynb",
|
||||
"language_code": "mo"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
|
|
@ -0,0 +1,85 @@
|
|||
# 初學者友善的 AI 範例
|
||||
|
||||
歡迎！此目錄包含簡單、獨立的範例，幫助您開始學習 AI 和機器學習。每個範例都設計為適合初學者，並附有詳細的註解和逐步解說。
|
||||
|
||||
## 📚 範例概覽
|
||||
|
||||
| 範例 | 描述 | 難度 | 先決條件 |
|
||||
|------|------|------|----------|
|
||||
| [Hello AI World](../../../examples/01-hello-ai-world.py) | 您的第一個 AI 程式 - 簡單的模式識別 | ⭐ 初學者 | Python 基礎 |
|
||||
| [Simple Neural Network](../../../examples/02-simple-neural-network.py) | 從零開始建立神經網絡 | ⭐⭐ 初學者+ | Python、基礎數學 |
|
||||
| [Image Classifier](./03-image-classifier.ipynb) | 使用預訓練模型進行圖像分類 | ⭐⭐ 初學者+ | Python、numpy |
|
||||
| [Text Sentiment](../../../examples/04-text-sentiment.py) | 分析文本情感（正面/負面） | ⭐⭐ 初學者+ | Python |
|
||||
|
||||
## 🚀 開始使用
|
||||
|
||||
### 先決條件
|
||||
|
||||
請確保您已安裝 Python（建議使用 3.8 或更高版本）。安裝所需的套件：
|
||||
|
||||
```bash
|
||||
# For Python scripts
|
||||
pip install numpy
|
||||
|
||||
# For Jupyter notebooks (image classifier)
|
||||
pip install jupyter numpy pillow tensorflow
|
||||
```
|
||||
|
||||
或者使用主課程中的 conda 環境：
|
||||
|
||||
```bash
|
||||
conda env create --name ai4beg --file ../environment.yml
|
||||
conda activate ai4beg
|
||||
```
|
||||
|
||||
### 執行範例
|
||||
|
||||
**對於 Python 腳本 (.py 檔案):**
|
||||
```bash
|
||||
python 01-hello-ai-world.py
|
||||
```
|
||||
|
||||
**對於 Jupyter 筆記本 (.ipynb 檔案):**
|
||||
```bash
|
||||
jupyter notebook 03-image-classifier.ipynb
|
||||
```
|
||||
|
||||
## 📖 學習路徑
|
||||
|
||||
我們建議按照以下順序進行學習：
|
||||
|
||||
1. **從 "Hello AI World" 開始** - 學習模式識別的基礎
|
||||
2. **建立簡單的神經網絡** - 理解神經網絡的運作方式
|
||||
3. **嘗試圖像分類器** - 使用真實圖像體驗 AI 的應用
|
||||
4. **分析文本情感** - 探索自然語言處理
|
||||
|
||||
## 💡 初學者提示
|
||||
|
||||
- **仔細閱讀程式碼註解** - 它們解釋了每行程式碼的作用
|
||||
- **多嘗試！** - 嘗試更改數值並觀察結果
|
||||
- **不用擔心完全理解** - 學習需要時間
|
||||
- **提出問題** - 使用 [討論板](https://github.com/microsoft/AI-For-Beginners/discussions)
|
||||
|
||||
## 🔗 下一步
|
||||
|
||||
完成這些範例後，探索完整課程：
|
||||
- [AI 簡介](../lessons/1-Intro/README.md)
|
||||
- [神經網絡](../lessons/3-NeuralNetworks/README.md)
|
||||
- [電腦視覺](../lessons/4-ComputerVision/README.md)
|
||||
- [自然語言處理](../lessons/5-NLP/README.md)
|
||||
|
||||
## 🤝 貢獻
|
||||
|
||||
覺得這些範例有幫助嗎？幫助我們改進：
|
||||
- 回報問題或提出改進建議
|
||||
- 添加更多適合初學者的範例
|
||||
- 改善文件和註解
|
||||
|
||||
---
|
||||
|
||||
*記住：每位專家都曾是初學者。祝您學習愉快！ 🎓*
|
||||
|
||||
---
|
||||
|
||||
**免責聲明**：
|
||||
本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。雖然我們致力於提供準確的翻譯，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵資訊，建議使用專業人工翻譯。我們對因使用此翻譯而產生的任何誤解或錯誤解釋不承擔責任。
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
# 給教育工作者
|
||||
|
||||
您是否希望在課堂上使用這份課程？請隨意使用！
|
||||
|
||||
事實上，您可以直接在 GitHub 上使用 GitHub Classroom 來進行。
|
||||
|
||||
要做到這一點，請先 fork 此 repo。您需要為每一課建立一個 repo，因此需要將每個文件夾提取到單獨的 repo 中。這樣，[GitHub Classroom](https://classroom.github.com/classrooms) 就能分別處理每一課。
|
||||
|
||||
這些[完整的指導說明](https://github.blog/2020-03-18-set-up-your-digital-classroom-with-github-classroom/)可以幫助您了解如何設置您的課堂。
|
||||
|
||||
## 直接使用此 repo
|
||||
|
||||
如果您希望直接使用目前的 repo，而不使用 GitHub Classroom，也可以這樣做。您需要與您的學生溝通，告訴他們一起學習哪一課。
|
||||
|
||||
在線上教學形式（例如 Zoom、Teams 或其他平台）中，您可以為測驗設置分組討論室，並指導學生做好學習準備。然後邀請學生參加測驗，並在指定時間以 "issues" 的形式提交答案。如果您希望學生公開合作，也可以用相同方式進行作業。
|
||||
|
||||
如果您更喜歡較為私密的形式，可以要求學生逐課 fork 此課程到他們自己的 GitHub 私人 repo，並授予您訪問權限。這樣他們可以私下完成測驗和作業，並通過 classroom repo 的 issues 提交給您。
|
||||
|
||||
在線課堂有許多不同的運作方式。請告訴我們哪種方式最適合您！
|
||||
|
||||
## 請提供您的意見
|
||||
|
||||
我們希望這份課程能真正幫助到您和您的學生。請在討論區提供您的反饋意見！
|
||||
|
||||
**免責聲明**：
|
||||
本文檔使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們努力確保準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始語言的文件應被視為權威來源。對於關鍵信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或誤釋不承擔責任。
|
||||
|
|
@ -0,0 +1,71 @@
|
|||
# 如何執行程式碼
|
||||
|
||||
本課程包含許多可執行的範例和實驗室，您會想要執行它們。為此，您需要能夠在本課程所提供的 Jupyter 筆記本中執行 Python 程式碼。您有幾種選擇可以執行程式碼：
|
||||
|
||||
## 在您電腦本地執行
|
||||
|
||||
要在您電腦本地執行程式碼，需要安裝 Python。建議安裝 **[miniconda](https://conda.io/en/latest/miniconda.html)** — 這是一個相對輕量的安裝，支援使用 `conda` 套件管理器管理不同的 Python **虛擬環境**。
|
||||
|
||||
安裝 miniconda 後，克隆此倉庫並建立一個虛擬環境供本課程使用：
|
||||
|
||||
```bash
|
||||
git clone http://github.com/microsoft/ai-for-beginners
|
||||
cd ai-for-beginners
|
||||
conda env create --name ai4beg --file .devcontainer/environment.yml
|
||||
conda activate ai4beg
|
||||
```
|
||||
|
||||
### 使用安裝了 Python 擴充功能的 Visual Studio Code
|
||||
|
||||
本課程最適合在安裝了 [Python 擴充功能](https://marketplace.visualstudio.com/items?itemName=ms-python.python&WT.mc_id=academic-77998-cacaste) 的 [Visual Studio Code](http://code.visualstudio.com/?WT.mc_id=academic-77998-cacaste) 中開啟進行使用。
|
||||
|
||||
> **注意**：當您克隆並在 VS Code 中開啟目錄時，它會自動建議您安裝 Python 擴充功能。您同時也需要依前述安裝 miniconda。
|
||||
|
||||
> **注意**：如果 VS Code 建議您在容器中重新開啟倉庫，您應該拒絕，以使用本地的 Python 安裝。
|
||||
|
||||
### 在瀏覽器中使用 Jupyter
|
||||
|
||||
您也可以在自己電腦的瀏覽器中使用 Jupyter 環境。傳統的 Jupyter 與 JupyterHub 都提供了方便的開發環境，包括自動補完、程式碼高亮等功能。
|
||||
|
||||
要在本地啟動 Jupyter，請前往課程目錄並執行：
|
||||
|
||||
```bash
|
||||
jupyter notebook
|
||||
```
|
||||
或
|
||||
```bash
|
||||
jupyterhub
|
||||
```
|
||||
然後您可以導航到任何 `.ipynb` 檔案，開啟並開始操作。
|
||||
|
||||
### 在容器中執行
|
||||
|
||||
另一種替代安裝 Python 的方法是使用容器來執行程式碼。由於我們的倉庫提供了一個特殊的 `.devcontainer` 資料夾，說明了如何為此倉庫建置容器，VS Code 提供機會讓您在容器中重新打開程式碼。這需要安裝 Docker，且流程較複雜，因此我們建議較有經驗的使用者採用此方式。
|
||||
|
||||
## 在雲端執行
|
||||
|
||||
如果您不想在本地安裝 Python，且能使用某些雲端資源，一個不錯的選擇是直接在雲端執行程式碼。您可以用以下幾種方法：
|
||||
|
||||
* 使用 **[GitHub Codespaces](https://github.com/features/codespaces)**，這是在 GitHub 上為您建立的虛擬環境，可透過 VS Code 瀏覽器介面存取。如果您有 Codespaces 的使用權，您只需點選倉庫中的 **Code** 按鈕，啟動 codespace，即可迅速開始。
|
||||
* 使用 **[Binder](https://mybinder.org/v2/gh/microsoft/ai-for-beginners/HEAD)**。[Binder](https://mybinder.org) 免費提供雲端計算資源，方便您測試 GitHub 上的程式碼。在首頁有個按鈕可在 Binder 中開啟此倉庫 — 這會快速導向 Binder 網站，並自動建構一個底層容器，順暢啟動 Jupyter 網頁介面。
|
||||
|
||||
> **注意**：為防止濫用，Binder 對部分網絡資源有限制，有時會阻擋某些從公共網際網路下載模型或資料集的程式碼，可能導致部分程式無法正常運行，您可能需要尋找替代方案。此外，Binder 提供的計算資源較基本，訓練過程會比較慢，尤其在後續較複雜課程中。
|
||||
|
||||
## 在雲端使用 GPU 執行
|
||||
|
||||
本課程後面的一些章節會大大受益於 GPU 支援。例如模型訓練，否則會非常緩慢。您可以選擇以下幾種方法，特別是您有透過 [Azure for Students](https://azure.microsoft.com/free/students/?WT.mc_id=academic-77998-cacaste) 或您的機構取得雲端資源：
|
||||
|
||||
* 建立 [Data Science Virtual Machine](https://docs.microsoft.com/learn/modules/intro-to-azure-data-science-virtual-machine/?WT.mc_id=academic-77998-cacaste) 並透過 Jupyter 連線。您可以直接將倉庫克隆到此虛擬機，開始學習。NC 系列虛擬機支援 GPU。
|
||||
|
||||
> **注意**：部分訂閱服務，包括 Azure for Students 預設不提供 GPU 支援，您可能需要透過技術支援申請額外 GPU 核心。
|
||||
|
||||
* 建立 [Azure Machine Learning Workspace](https://azure.microsoft.com/services/machine-learning/?WT.mc_id=academic-77998-cacaste) 並使用其 Notebook 功能。[此影片](https://azure-for-academics.github.io/quickstart/azureml-papers/) 示範如何將倉庫克隆到 Azure ML Notebook 並開始使用。
|
||||
|
||||
您亦可使用 Google Colab，它提供部分免費 GPU 支援，並能上傳 Jupyter 筆記本，一個一個地執行。
|
||||
|
||||
---
|
||||
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER START -->
|
||||
**免責聲明**：
|
||||
本文件經由 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們力求準確，請注意自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於重要資訊，建議使用專業人工翻譯。本公司不對使用本翻譯所引致的任何誤解或誤釋承擔責任。
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER END -->
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
# 開始使用此課程
|
||||
|
||||
## 你是學生嗎？
|
||||
|
||||
可以從以下資源開始：
|
||||
|
||||
* [學生中心頁面](https://docs.microsoft.com/learn/student-hub?WT.mc_id=academic-77998-cacaste) 在這個頁面，你可以找到初學者資源、學生套件，甚至有機會獲得免費認證券。這是一個值得收藏並定期查看的頁面，因為我們至少每月會更新內容。
|
||||
* [Microsoft 學生學習大使](https://studentambassadors.microsoft.com?WT.mc_id=academic-77998-cacaste) 加入一個全球性的學生大使社群，這可能是你進入 Microsoft 的途徑。
|
||||
|
||||
**學生們**，有幾種方式可以使用這個課程。首先，你可以直接在 GitHub 上閱讀文本並查看程式碼。如果你想在任何筆記本中執行程式碼，請[閱讀我們的指引](./how-to-run.md)，並在[這篇博客文章](https://soshnikov.com/education/how-to-execute-notebooks-from-github/)中找到更多建議。
|
||||
|
||||
> **注意**: [如何執行此課程中的程式碼的指引](./how-to-run.md)
|
||||
|
||||
## 自學
|
||||
|
||||
然而，如果你想將這門課程作為自學項目，我們建議你將整個倉庫 fork 到你自己的 GitHub 帳戶，並自行或與小組一起完成練習：
|
||||
|
||||
* 從課前測驗開始。
|
||||
* 閱讀課程的介紹文本。
|
||||
* 如果課程有額外的筆記本，請閱讀並執行其中的程式碼。如果同時提供 TensorFlow 和 PyTorch 筆記本，你可以專注於其中一個，選擇你喜歡的框架。
|
||||
* 筆記本通常包含一些挑戰，要求你稍微修改程式碼以進行實驗。
|
||||
* 完成課後測驗。
|
||||
* 如果模組附有實驗室，請完成作業。
|
||||
* 訪問[討論板](https://github.com/microsoft/AI-For-Beginners/discussions)來“公開學習”。
|
||||
|
||||
> 為了進一步學習，我們建議你跟隨這些 [Microsoft Learn](https://docs.microsoft.com/en-us/users/dmitrysoshnikov-9132/collections/31zgizg2p418yo/?WT.mc_id=academic-77998-cacaste) 模組和學習路徑。
|
||||
|
||||
**教師們**，我們已[包含一些建議](./for-teachers.md)來幫助你使用這個課程。
|
||||
|
||||
---
|
||||
|
||||
## 教學法
|
||||
|
||||
在構建這個課程時，我們選擇了兩個教學原則：確保它是**基於項目**的實踐性課程，並且包含**頻繁的測驗**。
|
||||
|
||||
通過確保內容與項目對應，學習過程對學生來說更具吸引力，並且能增強概念的記憶。此外，課前的低壓力測驗可以幫助學生集中注意力學習主題，而課後的第二次測驗則進一步鞏固記憶。這個課程設計靈活有趣，可以整體學習，也可以部分學習。項目從簡單開始，並在 12 週的學習周期結束時變得越來越複雜。
|
||||
|
||||
> **關於測驗的說明**: 所有測驗都包含在[這個應用程式](https://red-field-0a6ddfd03.1.azurestaticapps.net/)中，共有 50 個測驗，每個測驗包含三個問題。它們在課程中有連結，但測驗應用程式也可以在本地運行；請按照 `etc/quiz-app` 資料夾中的指引操作。
|
||||
|
||||
## 離線訪問
|
||||
|
||||
你可以使用 [Docsify](https://docsify.js.org/#/) 離線運行此文檔。Fork 此倉庫，在你的本地機器上[安裝 Docsify](https://docsify.js.org/#/quickstart)，然後在此倉庫的根目錄中輸入 `docsify serve`。網站將在本地端的 3000 埠上運行：`localhost:3000`。課程的 PDF 可在[此連結](../../../../../../../../../etc/pdf/readme.pdf)中獲取。
|
||||
|
||||
---
|
||||
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER START -->
|
||||
**免責聲明**：
|
||||
此文件已使用人工智能翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們致力於提供準確的翻譯，請注意自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於重要信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或誤釋不承擔責任。
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER END -->
|
||||
|
|
@ -0,0 +1,161 @@
|
|||
# 人工智能簡介
|
||||
|
||||
|
||||
|
||||
> 手繪筆記由 [Tomomi Imura](https://twitter.com/girlie_mac) 提供
|
||||
|
||||
## [課前測驗](https://ff-quizzes.netlify.app/en/ai/quiz/1)
|
||||
|
||||
**人工智能** 是一門令人興奮的科學學科，研究如何讓電腦表現出智能行為，例如執行人類擅長的事情。
|
||||
|
||||
最初，電腦是由 [查爾斯·巴貝奇](https://en.wikipedia.org/wiki/Charles_Babbage) 發明的，用於按照明確定義的程序（算法）處理數字。現代電腦雖然比19世紀提出的原始模型先進得多，但仍然遵循受控計算的理念。因此，如果我們知道實現目標所需的精確步驟序列，就可以編程讓電腦完成某些事情。
|
||||
|
||||
|
||||
|
||||
> 照片由 [Vickie Soshnikova](http://twitter.com/vickievalerie) 提供
|
||||
|
||||
> ✅ 從一張照片中判斷一個人的年齡是一項無法明確編程的任務，因為我們無法解釋當我們在腦海中得出一個數字時是如何做到的。
|
||||
|
||||
---
|
||||
|
||||
然而，有些任務我們並不知道如何明確解決。例如，從一張照片中判斷一個人的年齡。我們能夠做到這一點，是因為我們看過許多年齡不同的人的例子，但我們無法明確解釋我們是如何做到的，也無法編程讓電腦完成這項任務。這正是 **人工智能**（簡稱 AI）感興趣的任務類型。
|
||||
|
||||
✅ 想一想有哪些任務可以交給電腦處理，並且能從人工智能中受益。考慮金融、醫療和藝術領域——這些領域今天如何從人工智能中受益？
|
||||
|
||||
## 弱人工智能與強人工智能
|
||||
|
||||
弱人工智能 | 強人工智能
|
||||
---------------------------------------|-------------------------------------
|
||||
弱人工智能指的是為特定任務或一組狹窄任務設計和訓練的人工智能系統。|強人工智能，或稱人工通用智能（AGI），指的是具有與人類相當的智能和理解能力的人工智能系統。
|
||||
這些人工智能系統並不具備普遍的智能；它們在執行預定義的任務方面表現出色，但缺乏真正的理解或意識。|這些人工智能系統能夠執行人類可以完成的任何智力任務，適應不同的領域，並擁有某種形式的意識或自我認知。
|
||||
弱人工智能的例子包括像 Siri 或 Alexa 這樣的虛擬助手、流媒體服務使用的推薦算法，以及為特定客戶服務任務設計的聊天機器人。|實現強人工智能是人工智能研究的長期目標，這需要開發能夠在廣泛任務和情境中進行推理、學習、理解和適應的人工智能系統。
|
||||
弱人工智能高度專業化，並不具備人類般的認知能力或超出其狹窄領域的通用問題解決能力。|強人工智能目前仍是一個理論概念，尚無人工智能系統達到這種通用智能的水平。
|
||||
|
||||
更多資訊請參考 **[人工通用智能](https://en.wikipedia.org/wiki/Artificial_general_intelligence)** (AGI)。
|
||||
|
||||
## 智能的定義與圖靈測試
|
||||
|
||||
處理 **[智能](https://en.wikipedia.org/wiki/Intelligence)** 這個術語時的一個問題是，對於這個術語並沒有明確的定義。有人認為智能與 **抽象思維** 或 **自我意識** 有關，但我們無法準確定義它。
|
||||
|
||||
|
||||
|
||||
> [照片](https://unsplash.com/photos/75715CVEJhI) 由 [Amber Kipp](https://unsplash.com/@sadmax) 提供，來自 Unsplash
|
||||
|
||||
為了看出“智能”這個術語的模糊性，試著回答這個問題：“貓是智能的嗎？”不同的人往往會給出不同的答案，因為沒有普遍接受的測試來證明這一說法是真實的。如果你認為有——試著讓你的貓參加智商測試……
|
||||
|
||||
✅ 花點時間想一想你如何定義智能。一隻能解迷宮並獲得食物的烏鴉是智能的嗎？一個小孩是智能的嗎？
|
||||
|
||||
---
|
||||
|
||||
談到 AGI 時，我們需要某種方法來判斷我們是否創造了一個真正智能的系統。[艾倫·圖靈](https://en.wikipedia.org/wiki/Alan_Turing) 提出了一種方法，稱為 **[圖靈測試](https://en.wikipedia.org/wiki/Turing_test)**，這也可以作為智能的定義。該測試將給定系統與某些本質上智能的東西——一個真人進行比較，並且因為任何自動比較都可能被電腦程序繞過，我們使用一個人類審問者。因此，如果一個人無法在基於文本的對話中區分真人和電腦系統——該系統就被認為是智能的。
|
||||
|
||||
> 一個名為 [Eugene Goostman](https://en.wikipedia.org/wiki/Eugene_Goostman) 的聊天機器人於 2014 年在聖彼得堡接近通過圖靈測試，使用了一個巧妙的個性技巧。它一開始就聲明自己是一個 13 歲的烏克蘭男孩，這可以解釋知識的缺乏和文本中的一些不一致之處。該機器人在 5 分鐘的對話後說服了 30% 的評委相信它是人類，這是一個圖靈認為機器到 2000 年能夠通過的指標。然而，我們應該明白，這並不意味著我們已經創造了一個智能系統，或者電腦系統欺騙了人類審問者——實際上是機器人的創造者欺騙了人類！
|
||||
|
||||
✅ 你是否曾經被聊天機器人欺騙，認為你在與真人交談？它是如何說服你的？
|
||||
|
||||
## 人工智能的不同方法
|
||||
|
||||
如果我們希望電腦像人類一樣行事，我們需要以某種方式在電腦內部模擬我們的思維方式。因此，我們需要嘗試理解是什麼讓人類變得智能。
|
||||
|
||||
> 為了能夠將智能編程到機器中，我們需要了解我們自己做決策的過程是如何運作的。如果你進行一些自我反思，你會意識到有些過程是潛意識發生的——例如，我們可以在不思考的情況下區分貓和狗——而另一些則涉及推理。
|
||||
|
||||
解決這個問題有兩種可能的方法：
|
||||
|
||||
自上而下方法（符號推理） | 自下而上方法（神經網絡）
|
||||
---------------------------------------|-------------------------------------
|
||||
自上而下的方法模擬人類解決問題的推理方式。它涉及從人類中提取 **知識**，並以計算機可讀的形式表示。我們還需要開發一種方法來在計算機內部模擬 **推理**。|自下而上的方法模擬人腦的結構，由大量簡單的單元組成，稱為 **神經元**。每個神經元的作用類似於其輸入的加權平均值，我們可以通過提供 **訓練數據** 來訓練神經元網絡解決有用的問題。
|
||||
|
||||
此外，還有一些其他可能的智能方法：
|
||||
|
||||
* **湧現式**、**協同式** 或 **多代理方法** 基於這樣一個事實，即通過大量簡單代理的交互可以獲得複雜的智能行為。根據 [進化控制論](https://en.wikipedia.org/wiki/Global_brain#Evolutionary_cybernetics)，智能可以在 *元系統轉變* 的過程中從更簡單的反應行為中 *湧現*。
|
||||
|
||||
* **進化方法** 或 **遺傳算法** 是基於進化原則的優化過程。
|
||||
|
||||
我們將在課程後續部分考慮這些方法，但現在我們將重點放在兩個主要方向：自上而下和自下而上。
|
||||
|
||||
### 自上而下方法
|
||||
|
||||
在 **自上而下方法** 中，我們嘗試模擬我們的推理過程。由於我們在推理時可以跟隨自己的思路，我們可以嘗試將這一過程形式化並編程到計算機中。這被稱為 **符號推理**。
|
||||
|
||||
人們往往在頭腦中有一些指導其決策過程的規則。例如，當醫生診斷病人時，他或她可能意識到病人發燒，因此可能身體內部有某種炎症。通過將大量規則應用於特定問題，醫生可能最終得出診斷結論。
|
||||
|
||||
這種方法高度依賴於 **知識表示** 和 **推理**。從人類專家那裡提取知識可能是最困難的部分，因為醫生在許多情況下可能並不確切知道為什麼他或她會得出特定的診斷。有時解決方案只是自然而然地出現在他的腦海中，而沒有明確的思考。有些任務，例如從照片中判斷一個人的年齡，根本無法簡化為知識的操作。
|
||||
|
||||
### 自下而上方法
|
||||
|
||||
或者，我們可以嘗試模擬我們大腦中最簡單的元素——神經元。我們可以在計算機內部構建一個所謂的 **人工神經網絡**，然後通過提供例子來教它解決問題。這個過程類似於新生兒通過觀察學習周圍環境的方式。
|
||||
|
||||
✅ 做一些關於嬰兒如何學習的研究。嬰兒大腦的基本元素是什麼？
|
||||
|
||||
> | 那麼機器學習呢？ | |
|
||||
> |--------------|-----------|
|
||||
> | 基於計算機學習如何根據某些數據解決問題的人工智能部分稱為 **機器學習**。我們不會在本課程中考慮經典的機器學習——我們建議參考單獨的 [機器學習初學者課程](http://aka.ms/ml-beginners)。 |  |
|
||||
|
||||
## 人工智能的簡史
|
||||
|
||||
人工智能作為一個領域始於20世紀中葉。最初，符號推理是一種流行的方法，並且它帶來了一些重要的成功，例如專家系統——能夠在某些有限問題領域中充當專家的計算機程序。然而，很快就發現這種方法並不適用於大規模應用。從專家那裡提取知識、將其表示在計算機中並保持知識庫的準確性，事實證明這是一項非常複雜且在許多情況下成本過高的任務。這導致了20世紀70年代所謂的 [人工智能寒冬](https://en.wikipedia.org/wiki/AI_winter)。
|
||||
|
||||
<img alt="人工智能簡史" src="../../../../translated_images/zh-MO/history-of-ai.7e83efa70b537f5a.webp" width="70%"/>
|
||||
|
||||
> 圖片由 [Dmitry Soshnikov](http://soshnikov.com) 提供
|
||||
|
||||
隨著時間的推移，計算資源變得更便宜，並且有更多數據可用，因此神經網絡方法在許多領域（如計算機視覺或語音理解）中開始表現出色。在過去十年中，“人工智能”這個術語大多被用作神經網絡的同義詞，因為我們聽到的大多數人工智能成功案例都基於它們。
|
||||
|
||||
我們可以觀察到方法的變化，例如在創建下棋計算機程序時：
|
||||
|
||||
* 早期的下棋程序基於搜索——程序明確嘗試估計對手在給定的幾步棋中的可能走法，並根據幾步棋後可以達到的最佳位置選擇最佳走法。這導致了所謂的 [alpha-beta 剪枝](https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning) 搜索算法的發展。
|
||||
* 搜索策略在遊戲結束時效果很好，因為搜索空間受到少量可能走法的限制。然而，在遊戲開始時，搜索空間非常大，算法可以通過學習人類玩家之間的現有比賽來改進。隨後的實驗採用了所謂的 [基於案例的推理](https://en.wikipedia.org/wiki/Case-based_reasoning)，程序在知識庫中尋找與當前遊戲位置非常相似的案例。
|
||||
* 現代能夠擊敗人類玩家的程序基於神經網絡和 [強化學習](https://en.wikipedia.org/wiki/Reinforcement_learning)，程序通過長時間與自己對弈並從自己的錯誤中學習來學會下棋——這與人類學習下棋的方式非常相似。然而，計算機程序可以在更短的時間內進行更多的比賽，因此可以學得更快。
|
||||
|
||||
✅ 做一些研究，了解其他由人工智能參與的遊戲。
|
||||
|
||||
同樣，我們可以看到創建“會說話的程序”（可能通過圖靈測試）的方法如何改變：
|
||||
|
||||
* 這類早期程序（如 [Eliza](https://en.wikipedia.org/wiki/ELIZA)）基於非常簡單的語法規則和將輸入句子重新表述為問題。
|
||||
* 現代助手（如 Cortana、Siri 或 Google Assistant）都是混合系統，使用神經網絡將語音轉換為文本並識別我們的意圖，然後使用一些推理或明確的算法執行所需的操作。
|
||||
* 未來，我們可能會期待一個完全基於神經網絡的模型能夠自行處理對話。最近的 GPT 和 [Turing-NLG](https://www.microsoft.com/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft) 神經網絡家族在這方面表現出色。
|
||||
|
||||
<img alt="圖靈測試的演變" src="../../../../translated_images/zh-MO/turing-test-evol.4184696701293ead.webp" width="70%"/>
|
||||
> 圖片由 Dmitry Soshnikov 提供，[照片](https://unsplash.com/photos/r8LmVbUKgns) 由 [Marina Abrosimova](https://unsplash.com/@abrosimova_marina_foto) 提供，Unsplash
|
||||
|
||||
## 最近的人工智能研究
|
||||
|
||||
神經網絡研究的巨大增長始於 2010 年左右，當時大型公共數據集開始變得可用。一個名為 [ImageNet](https://en.wikipedia.org/wiki/ImageNet) 的龐大圖像集合，包含約 1400 萬張帶註解的圖像，催生了 [ImageNet 大規模視覺識別挑戰賽](https://image-net.org/challenges/LSVRC/)。
|
||||
|
||||
|
||||
|
||||
> 圖片由 [Dmitry Soshnikov](http://soshnikov.com) 提供
|
||||
|
||||
2012 年，[卷積神經網絡](../4-ComputerVision/07-ConvNets/README.md) 首次被用於圖像分類，這導致分類錯誤率顯著下降（從接近 30% 降至 16.4%）。2015 年，微軟研究院的 ResNet 架構[達到了人類水平的準確率](https://doi.org/10.1109/ICCV.2015.123)。
|
||||
|
||||
自此以來，神經網絡在許多任務中展現了非常成功的表現：
|
||||
|
||||
---
|
||||
|
||||
年份 | 達到人類水平的任務
|
||||
-----|--------
|
||||
2015 | [圖像分類](https://doi.org/10.1109/ICCV.2015.123)
|
||||
2016 | [對話語音識別](https://arxiv.org/abs/1610.05256)
|
||||
2018 | [自動機器翻譯](https://arxiv.org/abs/1803.05567)（中譯英）
|
||||
2020 | [圖像標註](https://arxiv.org/abs/2009.13682)
|
||||
|
||||
在過去幾年中，我們見證了大型語言模型的巨大成功，例如 BERT 和 GPT-3。這主要是因為有大量的通用文本數據可用，這使我們能夠訓練模型來捕捉文本的結構和意義，先在通用文本集合上進行預訓練，然後再將這些模型專門化用於更具體的任務。我們將在本課程稍後學習更多關於[自然語言處理](../5-NLP/README.md)的內容。
|
||||
|
||||
## 🚀 挑戰
|
||||
|
||||
在互聯網上進行探索，判斷您認為人工智能在哪些領域最有效地被使用。是地圖應用程式、語音轉文字服務還是電子遊戲？研究該系統是如何構建的。
|
||||
|
||||
## [課後測驗](https://ff-quizzes.netlify.app/en/ai/quiz/2)
|
||||
|
||||
## 回顧與自學
|
||||
|
||||
通過閱讀[這一課程](https://github.com/microsoft/ML-For-Beginners/tree/main/1-Introduction/2-history-of-ML)來回顧人工智能和機器學習的歷史。從該課程或本課程頂部的手繪筆記中選取一個元素，深入研究以了解其演變的文化背景。
|
||||
|
||||
**作業**：[遊戲創作比賽](assignment.md)
|
||||
|
||||
---
|
||||
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER START -->
|
||||
**免責聲明**：
|
||||
此文件已使用人工智能翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們致力於提供準確的翻譯，請注意自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於重要信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或誤釋不承擔責任。
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER END -->
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
# 遊戲創作比賽
|
||||
|
||||
遊戲是一個受到人工智能（AI）和機器學習（ML）發展深刻影響的領域。在這次作業中，請撰寫一篇短文，討論一款您喜歡的遊戲，該遊戲因人工智能的演變而受到影響。這款遊戲應該足夠古老，以便能夠受到多種類型的計算機處理系統的影響。一個不錯的例子是象棋或圍棋，但也可以看看像乒乓球或吃豆人這樣的電子遊戲。撰寫一篇文章，探討該遊戲的過去、現在以及人工智能的未來。
|
||||
|
||||
---
|
||||
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER START -->
|
||||
**免責聲明**：
|
||||
此文件使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。我們致力於提供準確的翻譯，但請注意，自動翻譯可能包含錯誤或不準確之處。應以原文文件作為權威來源。對於關鍵資訊，建議尋求專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或誤釋不承擔責任。
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER END -->
|
||||
|
|
@ -0,0 +1,477 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"# 實作動物專家系統\n",
|
||||
"\n",
|
||||
"取自 [AI for Beginners Curriculum](http://github.com/microsoft/ai-for-beginners) 的一個範例。\n",
|
||||
"\n",
|
||||
"在本範例中，我們將實作一個簡單的知識庫系統，用來根據一些物理特徵判定動物。系統可以用以下的 AND-OR 樹來表示（這是整棵樹的一部分，我們可以輕易加入更多規則）：\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 我們自己的專家系統外殼與逆向推理\n",
|
||||
"\n",
|
||||
"讓我們嘗試定義一種基於產生規則的簡單知識表示語言。我們將使用 Python 類作為關鍵字來定義規則。基本上會有三種類型的類別：\n",
|
||||
"* `Ask` 代表一個需要向使用者提問的問題。它包含可能答案的集合。\n",
|
||||
"* `If` 代表一條規則，它只是儲存規則內容的語法糖。\n",
|
||||
"* `AND`/`OR` 是表示樹狀結構中 AND/OR 分支的類別。它們只是儲存內部參數的列表。為了簡化程式碼，所有功能都定義在父類別 `Content` 中。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class Ask():\n",
|
||||
" def __init__(self,choices=['y','n']):\n",
|
||||
" self.choices = choices\n",
|
||||
" def ask(self):\n",
|
||||
" if max([len(x) for x in self.choices])>1:\n",
|
||||
" for i,x in enumerate(self.choices):\n",
|
||||
" print(\"{0}. {1}\".format(i,x),flush=True)\n",
|
||||
" x = int(input())\n",
|
||||
" return self.choices[x]\n",
|
||||
" else:\n",
|
||||
" print(\"/\".join(self.choices),flush=True)\n",
|
||||
" return input()\n",
|
||||
"\n",
|
||||
"class Content():\n",
|
||||
" def __init__(self,x):\n",
|
||||
" self.x=x\n",
|
||||
" \n",
|
||||
"class If(Content):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"class AND(Content):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"class OR(Content):\n",
|
||||
" pass"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在我們的系統中，工作記憶會包含作為**屬性-值對**的**事實**清單。知識庫可以定義為一個大型字典，將動作（應該插入工作記憶的新事實）映射到條件，這些條件以 AND-OR 表達式表示。此外，某些事實可以被 `Ask`。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"rules = {\n",
|
||||
" 'default': Ask(['y','n']),\n",
|
||||
" 'color' : Ask(['red-brown','black and white','other']),\n",
|
||||
" 'pattern' : Ask(['dark stripes','dark spots']),\n",
|
||||
" 'mammal': If(OR(['hair','gives milk'])),\n",
|
||||
" 'carnivor': If(OR([AND(['sharp teeth','claws','forward-looking eyes']),'eats meat'])),\n",
|
||||
" 'ungulate': If(['mammal',OR(['has hooves','chews cud'])]),\n",
|
||||
" 'bird': If(OR(['feathers',AND(['flies','lies eggs'])])),\n",
|
||||
" 'animal:monkey' : If(['mammal','carnivor','color:red-brown','pattern:dark spots']),\n",
|
||||
" 'animal:tiger' : If(['mammal','carnivor','color:red-brown','pattern:dark stripes']),\n",
|
||||
" 'animal:giraffe' : If(['ungulate','long neck','long legs','pattern:dark spots']),\n",
|
||||
" 'animal:zebra' : If(['ungulate','pattern:dark stripes']),\n",
|
||||
" 'animal:ostrich' : If(['bird','long nech','color:black and white','cannot fly']),\n",
|
||||
" 'animal:pinguin' : If(['bird','swims','color:black and white','cannot fly']),\n",
|
||||
" 'animal:albatross' : If(['bird','flies well'])\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"為執行逆向推理，我們將定義 `Knowledgebase` 類別。它將包含：\n",
|
||||
"* 工作的 `memory` - 一個將屬性映射到值的字典\n",
|
||||
"* 以上述格式定義的知識庫 `rules`\n",
|
||||
"\n",
|
||||
"兩個主要方法為：\n",
|
||||
"* `get` 用於獲取屬性的值，必要時執行推理。例如，`get('color')` 會取得顏色槽的值（必要時會詢問，並將值儲存在工作記憶體中以供日後使用）。如果我們詢問 `get('color:blue')`，它會詢問顏色，然後根據顏色返回 `y`/`n` 的值。\n",
|
||||
"* `eval` 執行實際的推理，即遍歷 AND/OR 樹、評估子目標，等等。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class KnowledgeBase():\n",
|
||||
" def __init__(self,rules):\n",
|
||||
" self.rules = rules\n",
|
||||
" self.memory = {}\n",
|
||||
" \n",
|
||||
" def get(self,name):\n",
|
||||
" if ':' in name:\n",
|
||||
" k,v = name.split(':')\n",
|
||||
" vv = self.get(k)\n",
|
||||
" return 'y' if v==vv else 'n'\n",
|
||||
" if name in self.memory.keys():\n",
|
||||
" return self.memory[name]\n",
|
||||
" for fld in self.rules.keys():\n",
|
||||
" if fld==name or fld.startswith(name+\":\"):\n",
|
||||
" # print(\" + proving {}\".format(fld))\n",
|
||||
" value = 'y' if fld==name else fld.split(':')[1]\n",
|
||||
" res = self.eval(self.rules[fld],field=name)\n",
|
||||
" if res!='y' and res!='n' and value=='y':\n",
|
||||
" self.memory[name] = res\n",
|
||||
" return res\n",
|
||||
" if res=='y':\n",
|
||||
" self.memory[name] = value\n",
|
||||
" return value\n",
|
||||
" # field is not found, using default\n",
|
||||
" res = self.eval(self.rules['default'],field=name)\n",
|
||||
" self.memory[name]=res\n",
|
||||
" return res\n",
|
||||
" \n",
|
||||
" def eval(self,expr,field=None):\n",
|
||||
" # print(\" + eval {}\".format(expr))\n",
|
||||
" if isinstance(expr,Ask):\n",
|
||||
" print(field)\n",
|
||||
" return expr.ask()\n",
|
||||
" elif isinstance(expr,If):\n",
|
||||
" return self.eval(expr.x)\n",
|
||||
" elif isinstance(expr,AND) or isinstance(expr,list):\n",
|
||||
" expr = expr.x if isinstance(expr,AND) else expr\n",
|
||||
" for x in expr:\n",
|
||||
" if self.eval(x)=='n':\n",
|
||||
" return 'n'\n",
|
||||
" return 'y'\n",
|
||||
" elif isinstance(expr,OR):\n",
|
||||
" for x in expr.x:\n",
|
||||
" if self.eval(x)=='y':\n",
|
||||
" return 'y'\n",
|
||||
" return 'n'\n",
|
||||
" elif isinstance(expr,str):\n",
|
||||
" return self.get(expr)\n",
|
||||
" else:\n",
|
||||
" print(\"Unknown expr: {}\".format(expr))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"現在讓我們定義我們的動物知識庫並進行諮詢。請注意，此過程會向您提問。對於是非問題，您可以輸入 `y`/`n` 回答，對於有多個選項的問題，您可以輸入數字（0..N）來選擇答案。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"hair\n",
|
||||
"y/n\n",
|
||||
"sharp teeth\n",
|
||||
"y/n\n",
|
||||
"claws\n",
|
||||
"y/n\n",
|
||||
"forward-looking eyes\n",
|
||||
"y/n\n",
|
||||
"color\n",
|
||||
"0. red-brown\n",
|
||||
"1. black and white\n",
|
||||
"2. other\n",
|
||||
"has hooves\n",
|
||||
"y/n\n",
|
||||
"long neck\n",
|
||||
"y/n\n",
|
||||
"long legs\n",
|
||||
"y/n\n",
|
||||
"pattern\n",
|
||||
"0. dark stripes\n",
|
||||
"1. dark spots\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"'giraffe'"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"kb = KnowledgeBase(rules)\n",
|
||||
"kb.get('animal')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 使用 Experta 進行前向推論\n",
|
||||
"\n",
|
||||
"在下一個範例中，我們將嘗試使用其中一個知識表示庫 [Experta](https://github.com/nilp0inter/experta) 來實現前向推論。**Experta** 是一個用於在 Python 中創建前向推論系統的庫，其設計與經典舊系統 [CLIPS](http://www.clipsrules.net/index.html) 類似。\n",
|
||||
"\n",
|
||||
"我們本來也可以自行實現前向鏈式推論，並不會有太大問題，但天真的實現通常效率不高。為了更有效地進行規則匹配，使用了一種特殊算法 [Rete](https://en.wikipedia.org/wiki/Rete_algorithm)。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting git+https://github.com/nilp0inter/experta\n",
|
||||
" Cloning https://github.com/nilp0inter/experta to /tmp/pip-req-build-7qurtwk3\n",
|
||||
" Running command git clone --filter=blob:none --quiet https://github.com/nilp0inter/experta /tmp/pip-req-build-7qurtwk3\n",
|
||||
" Resolved https://github.com/nilp0inter/experta to commit c6d5834b123861f5ae09e7d07027dc98bec58741\n",
|
||||
" Installing build dependencies ... \u001b[?25ldone\n",
|
||||
"\u001b[?25h Getting requirements to build wheel ... \u001b[?25ldone\n",
|
||||
"\u001b[?25h Preparing metadata (pyproject.toml) ... \u001b[?25ldone\n",
|
||||
"\u001b[?25hRequirement already satisfied: frozendict~=2.4.6 in /opt/conda/envs/ai4beg/lib/python3.12/site-packages (from experta==1.9.5.dev1) (2.4.7)\n",
|
||||
"Collecting schema~=0.6.7 (from experta==1.9.5.dev1)\n",
|
||||
" Downloading schema-0.6.8-py2.py3-none-any.whl.metadata (14 kB)\n",
|
||||
"Downloading schema-0.6.8-py2.py3-none-any.whl (14 kB)\n",
|
||||
"Building wheels for collected packages: experta\n",
|
||||
" Building wheel for experta (pyproject.toml) ... \u001b[?25ldone\n",
|
||||
"\u001b[?25h Created wheel for experta: filename=experta-1.9.5.dev1-py3-none-any.whl size=34804 sha256=888c459512a5e713f4b674caa9a0f96cfdf07ec0d6eb56cc318ce0653d218014\n",
|
||||
" Stored in directory: /tmp/pip-ephem-wheel-cache-1eeii9zy/wheels/3d/e8/bb/22d7956359603fa8dd679aa09f5b8efb3f29991c3986fdc787\n",
|
||||
"Successfully built experta\n",
|
||||
"Installing collected packages: schema, experta\n",
|
||||
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/2\u001b[0m [experta]\n",
|
||||
"\u001b[1A\u001b[2KSuccessfully installed experta-1.9.5.dev1 schema-0.6.8\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"!{sys.executable} -m pip install git+https://github.com/nilp0inter/experta"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from experta import *\n",
|
||||
"#import experta"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我們將定義我們的系統為一個繼承自 `KnowledgeEngine` 的類別。每條規則由一個帶有 `@Rule` 註解的獨立函數定義，該註解指定了規則應該在何時觸發。在規則內，我們可以使用 `declare` 函數新增事實，新增這些事實將導致前向推理引擎調用更多規則。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class Animals(KnowledgeEngine):\n",
|
||||
" @Rule(OR(\n",
|
||||
" AND(Fact('sharp teeth'),Fact('claws'),Fact('forward looking eyes')),\n",
|
||||
" Fact('eats meat')))\n",
|
||||
" def cornivor(self):\n",
|
||||
" self.declare(Fact('carnivor'))\n",
|
||||
" \n",
|
||||
" @Rule(OR(Fact('hair'),Fact('gives milk')))\n",
|
||||
" def mammal(self):\n",
|
||||
" self.declare(Fact('mammal'))\n",
|
||||
"\n",
|
||||
" @Rule(Fact('mammal'),\n",
|
||||
" OR(Fact('has hooves'),Fact('chews cud')))\n",
|
||||
" def hooves(self):\n",
|
||||
" self.declare('ungulate')\n",
|
||||
" \n",
|
||||
" @Rule(OR(Fact('feathers'),AND(Fact('flies'),Fact('lays eggs'))))\n",
|
||||
" def bird(self):\n",
|
||||
" self.declare('bird')\n",
|
||||
" \n",
|
||||
" @Rule(Fact('mammal'),Fact('carnivor'),\n",
|
||||
" Fact(color='red-brown'),\n",
|
||||
" Fact(pattern='dark spots'))\n",
|
||||
" def monkey(self):\n",
|
||||
" self.declare(Fact(animal='monkey'))\n",
|
||||
"\n",
|
||||
" @Rule(Fact('mammal'),Fact('carnivor'),\n",
|
||||
" Fact(color='red-brown'),\n",
|
||||
" Fact(pattern='dark stripes'))\n",
|
||||
" def tiger(self):\n",
|
||||
" self.declare(Fact(animal='tiger'))\n",
|
||||
"\n",
|
||||
" @Rule(Fact('ungulate'),\n",
|
||||
" Fact('long neck'),\n",
|
||||
" Fact('long legs'),\n",
|
||||
" Fact(pattern='dark spots'))\n",
|
||||
" def giraffe(self):\n",
|
||||
" self.declare(Fact(animal='giraffe'))\n",
|
||||
"\n",
|
||||
" @Rule(Fact('ungulate'),\n",
|
||||
" Fact(pattern='dark stripes'))\n",
|
||||
" def zebra(self):\n",
|
||||
" self.declare(Fact(animal='zebra'))\n",
|
||||
"\n",
|
||||
" @Rule(Fact('bird'),\n",
|
||||
" Fact('long neck'),\n",
|
||||
" Fact('cannot fly'),\n",
|
||||
" Fact(color='black and white'))\n",
|
||||
" def straus(self):\n",
|
||||
" self.declare(Fact(animal='ostrich'))\n",
|
||||
"\n",
|
||||
" @Rule(Fact('bird'),\n",
|
||||
" Fact('swims'),\n",
|
||||
" Fact('cannot fly'),\n",
|
||||
" Fact(color='black and white'))\n",
|
||||
" def pinguin(self):\n",
|
||||
" self.declare(Fact(animal='pinguin'))\n",
|
||||
"\n",
|
||||
" @Rule(Fact('bird'),\n",
|
||||
" Fact('flies well'))\n",
|
||||
" def albatros(self):\n",
|
||||
" self.declare(Fact(animal='albatross'))\n",
|
||||
" \n",
|
||||
" @Rule(Fact(animal=MATCH.a))\n",
|
||||
" def print_result(self,a):\n",
|
||||
" print('Animal is {}'.format(a))\n",
|
||||
" \n",
|
||||
" def factz(self,l):\n",
|
||||
" for x in l:\n",
|
||||
" self.declare(x)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"一旦我們定義了一個知識庫，我們就會用一些初始事實填充我們的工作記憶，然後調用 `run()` 方法來執行推理。你可以看到結果中新的推斷事實被添加到工作記憶中，包括關於動物的最終事實（如果我們正確設置了所有初始事實）。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Animal is tiger\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"FactList([(0, InitialFact()),\n",
|
||||
" (1, Fact(color='red-brown')),\n",
|
||||
" (2, Fact(pattern='dark stripes')),\n",
|
||||
" (3, Fact('sharp teeth')),\n",
|
||||
" (4, Fact('claws')),\n",
|
||||
" (5, Fact('forward looking eyes')),\n",
|
||||
" (6, Fact('gives milk')),\n",
|
||||
" (7, Fact('mammal')),\n",
|
||||
" (8, Fact('carnivor')),\n",
|
||||
" (9, Fact(animal='tiger'))])"
|
||||
]
|
||||
},
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"ex1 = Animals()\n",
|
||||
"ex1.reset()\n",
|
||||
"ex1.factz([\n",
|
||||
" Fact(color='red-brown'),\n",
|
||||
" Fact(pattern='dark stripes'),\n",
|
||||
" Fact('sharp teeth'),\n",
|
||||
" Fact('claws'),\n",
|
||||
" Fact('forward looking eyes'),\n",
|
||||
" Fact('gives milk')])\n",
|
||||
"ex1.run()\n",
|
||||
"ex1.facts"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n\n<!-- CO-OP TRANSLATOR DISCLAIMER START -->\n**免責聲明**： \n本文件乃使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 所翻譯。儘管我們力求準確，但請注意自動翻譯可能存在錯誤或不準確之處。原始文件的母語版本應視為權威來源。對於重要資訊，建議採用專業人工翻譯。我們對因使用本翻譯而導致的任何誤解或誤譯概不負責。\n<!-- CO-OP TRANSLATOR DISCLAIMER END -->\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.7.4 64-bit (conda)",
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5"
|
||||
}
|
||||
},
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.2"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "8ef43db4b9182239fd150a76bd494fdb",
|
||||
"translation_date": "2026-01-15T11:43:41+00:00",
|
||||
"source_file": "lessons/2-Symbolic/Animals.ipynb",
|
||||
"language_code": "mo"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,593 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"# 家庭關係本體\n",
|
||||
"\n",
|
||||
"此範例是 [AI for Beginners Curriculum](http://github.com/microsoft/ai-for-beginners) 的一部分，靈感來自 [這篇部落格文章](https://habr.com/post/270857/)。\n",
|
||||
"\n",
|
||||
"我總是覺得記住家庭中人與人之間的不同關係很困難。在這個範例中，我們將使用一個定義家庭關係的本體，以及實際的家譜樹，並展示如何進行自動推理以找出所有親屬。\n",
|
||||
"\n",
|
||||
"### 獲取家譜樹\n",
|
||||
"\n",
|
||||
"作為範例，我們將使用 [羅曼諾夫沙皇家族](https://en.wikipedia.org/wiki/House_of_Romanov) 的家譜樹。描述家庭關係最常用的格式是 [GEDCOM](https://en.wikipedia.org/wiki/GEDCOM)。我們將使用 GEDCOM 格式的羅曼諾夫家族樹：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0 HEAD\n",
|
||||
"1 CHAR UTF8\n",
|
||||
"1 GEDC\n",
|
||||
"2 VERS 5.5\n",
|
||||
"0 @0@ INDI\n",
|
||||
"1 NAME Mihail Fedorovich /Romanov/\n",
|
||||
"1 SEX M\n",
|
||||
"1 BIRT\n",
|
||||
"2 DATE 1613\n",
|
||||
"1 DEAT \n",
|
||||
"2 DATE 1645\n",
|
||||
"1 FAMS @41@\n",
|
||||
"0 @1@ INDI\n",
|
||||
"1 NAME Evdokija Lukjanovna /Streshneva/\n",
|
||||
"1 SEX F\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!head -15 data/tsars.ged"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"要使用 GEDCOM 文件，我們可以使用 `python-gedcom` 庫：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting python-gedcom\n",
|
||||
" Downloading python_gedcom-1.0.0-py2.py3-none-any.whl (35 kB)\n",
|
||||
"Installing collected packages: python-gedcom\n",
|
||||
"Successfully installed python-gedcom-1.0.0\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"!{sys.executable} -m pip install python-gedcom"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"此庫消除了文件解析的一些技術問題，但仍然為我們提供了對樹中所有個人和家庭的相當低層次的訪問。以下是我們如何解析文件並顯示所有個人列表的方法：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from gedcom.parser import Parser\n",
|
||||
"from gedcom.element.individual import IndividualElement\n",
|
||||
"from gedcom.element.family import FamilyElement\n",
|
||||
"g = Parser()\n",
|
||||
"g.parse_file('data/tsars.ged')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"scrolled": true,
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[('@0@', ('Mihail Fedorovich', 'Romanov')),\n",
|
||||
" ('@1@', ('Evdokija Lukjanovna', 'Streshneva')),\n",
|
||||
" ('@2@', ('Aleksej Mihajlovich', 'Romanov')),\n",
|
||||
" ('@3@', ('Marija Ilinichna', 'Miloslavskaja')),\n",
|
||||
" ('@4@', ('Natalja Kirillovna', 'Naryshkina')),\n",
|
||||
" ('@5@', ('Marfa Matveevna', 'Apraksina')),\n",
|
||||
" ('@6@', ('Fedor Alekseevich', 'Romanov')),\n",
|
||||
" ('@7@', ('Sofja Aleksevna', 'Romanova')),\n",
|
||||
" ('@8@', ('Ivan V Alekseevich', 'Romanov')),\n",
|
||||
" ('@9@', ('Praskovja Fedorovna', 'Saltykova')),\n",
|
||||
" ('@10@', ('Ekaterina Ivanovna', 'Romanova')),\n",
|
||||
" ('@11@', ('Anna Ivanovna', 'Romanova')),\n",
|
||||
" ('@12@', ('Fridrih Vilgelm', 'Kurlandskij')),\n",
|
||||
" ('@13@', ('Karl Leopold', 'Meklenburg-Shverinskij')),\n",
|
||||
" ('@14@', ('Anna Leopoldovna', 'Meklenburg-Shverinskaja')),\n",
|
||||
" ('@15@', ('Anton Ulrih', 'Braunshvejg-Volfenbjuttelskij')),\n",
|
||||
" ('@16@', ('Ivan VI Antonovich', 'Braunshvejg-Volfenbjuttelskij')),\n",
|
||||
" ('@17@', ('Petr I Alekseevich', 'Romanov')),\n",
|
||||
" ('@18@', ('Evdokija Fedorovna', 'Lopuhina')),\n",
|
||||
" ('@19@', ('Ekaterina I Alekseevna', 'Mihajlova')),\n",
|
||||
" ('@20@', ('Aleksej Petrovich', 'Romanov')),\n",
|
||||
" ('@21@', ('Sharlotta Kristina', 'Braunshvejg-Volfenbjuttelskaja')),\n",
|
||||
" ('@22@', ('Petr II Alekseevich', 'Romanov')),\n",
|
||||
" ('@23@', ('Anna Petrovna', 'Romanova')),\n",
|
||||
" ('@24@', ('Elizaveta Petrovna', 'Romanova')),\n",
|
||||
" ('@25@', ('Karl Fridrih', 'Golshtejn-Gottorpskij')),\n",
|
||||
" ('@26@', ('Petr III Fedorovich', 'Romanov')),\n",
|
||||
" ('@27@', ('Ekaterina II', 'Alekseevna')),\n",
|
||||
" ('@28@', ('Pavel I Petrovich', 'Romanov')),\n",
|
||||
" ('@29@', ('Natalja Alekseevna', 'Gessen-Darmshtadskaja')),\n",
|
||||
" ('@30@', ('Marija Fedorovna', 'Vjurtembergskaja')),\n",
|
||||
" ('@31@', ('Aleksandr I Pavlovich', 'Romanov')),\n",
|
||||
" ('@32@', ('Elizaveta Alekseevna', 'Baden-Durlahskaja')),\n",
|
||||
" ('@33@', ('Nikolaj I Pavlovich', 'Romanov')),\n",
|
||||
" ('@34@', ('Aleksandra Fedorovna', 'Prusskaja')),\n",
|
||||
" ('@35@', ('Aleksandr II Nikolaevich', 'Romanov')),\n",
|
||||
" ('@36@', ('Marija Aleksandrovna', 'Gessenskaja')),\n",
|
||||
" ('@37@', ('Aleksandr III Aleksandrovich', 'Romanov')),\n",
|
||||
" ('@38@', ('Marija Fedorovna', 'Datskaja')),\n",
|
||||
" ('@39@', ('Nikolaj II Aleksandrovich', 'Romanov')),\n",
|
||||
" ('@40@', ('Aleksandra Fedorovna', 'Gessenskaja'))]"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"d = g.get_element_dictionary()\n",
|
||||
"[ (k,v.get_name()) for k,v in d.items() if isinstance(v,IndividualElement)]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"以下是我們如何獲取有關家庭的信息。請注意，這會給我們一個**標識符**的列表，如果我們想要更清楚的內容，我們需要將它們轉換為名稱：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[('@41@', ['@0@', '@1@', '@2@']),\n",
|
||||
" ('@42@', ['@2@', '@3@', '@6@', '@7@', '@8@']),\n",
|
||||
" ('@43@', ['@8@', '@9@', '@10@', '@11@']),\n",
|
||||
" ('@44@', ['@13@', '@10@', '@14@']),\n",
|
||||
" ('@45@', ['@15@', '@14@', '@16@']),\n",
|
||||
" ('@46@', ['@2@', '@4@', '@17@']),\n",
|
||||
" ('@47@', ['@17@', '@18@', '@20@']),\n",
|
||||
" ('@48@', ['@20@', '@21@', '@22@']),\n",
|
||||
" ('@49@', ['@17@', '@19@', '@23@', '@24@']),\n",
|
||||
" ('@50@', ['@25@', '@23@', '@26@']),\n",
|
||||
" ('@51@', ['@26@', '@27@', '@28@']),\n",
|
||||
" ('@52@', ['@28@', '@30@', '@31@', '@33@']),\n",
|
||||
" ('@53@', ['@33@', '@34@', '@35@']),\n",
|
||||
" ('@54@', ['@35@', '@36@', '@37@']),\n",
|
||||
" ('@55@', ['@37@', '@38@', '@39@'])]"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"d = g.get_element_dictionary()\n",
|
||||
"[ (k,[x.get_value() for x in v.get_child_elements()]) for k,v in d.items() if isinstance(v,FamilyElement)]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 獲取家庭本體\n",
|
||||
"\n",
|
||||
"接下來，我們來看看[家庭本體](https://raw.githubusercontent.com/blokhin/genealogical-trees/master/data/header.ttl)，它被定義為一組語義網三元組。這個本體定義了像 `isUncleOf`、`isCousinOf` 等許多關係。所有這些關係都是基於基本謂詞 `isMotherOf`、`isFatherOf`、`isBrotherOf` 和 `isSisterOf` 來定義的。我們將使用自動推理來根據本體推導出所有其他關係。\n",
|
||||
"\n",
|
||||
"以下是 `isAuntOf` 屬性的範例定義，它被定義為 `isSisterOf` 和 `isParentOf` 的組合（*姑姑是某人父母的姐妹*）。\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"fhkb:isAuntOf a owl:ObjectProperty ;\n",
|
||||
" rdfs:domain fhkb:Woman ;\n",
|
||||
" rdfs:range fhkb:Person ;\n",
|
||||
" owl:propertyChainAxiom ( fhkb:isSisterOf fhkb:isParentOf ) .\n",
|
||||
"```\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"@prefix fhkb: <http://www.example.com/genealogy.owl#> .\n",
|
||||
"@prefix owl: <http://www.w3.org/2002/07/owl#> .\n",
|
||||
"@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .\n",
|
||||
"@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n",
|
||||
"@prefix xml: <http://www.w3.org/XML/1998/namespace> .\n",
|
||||
"@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .\n",
|
||||
"\n",
|
||||
"<http://www.example.com/genealogy.owl#> a owl:Ontology .\n",
|
||||
"\n",
|
||||
"fhkb:DomainEntity a owl:Class .\n",
|
||||
"\n",
|
||||
"fhkb:Man a owl:Class ;\n",
|
||||
" owl:equivalentClass [ a owl:Class ;\n",
|
||||
" owl:intersectionOf ( fhkb:Person [ a owl:Restriction ;\n",
|
||||
" owl:onProperty fhkb:hasSex ;\n",
|
||||
" owl:someValuesFrom fhkb:Male ] ) ] .\n",
|
||||
"\n",
|
||||
"fhkb:Woman a owl:Class ;\n",
|
||||
" owl:equivalentClass [ a owl:Class ;\n",
|
||||
" owl:intersectionOf ( fhkb:Person [ a owl:Restriction ;\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!head -20 data/onto.ttl"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 建構推理用本體\n",
|
||||
"\n",
|
||||
"為了簡化，我們將建立一個本體檔案，該檔案將包含來自家庭本體的原始規則，以及來自我們 GEDCOM 檔案的個人事實。我們會逐一檢視 GEDCOM 檔案，提取有關家庭和個人的資訊，並將其轉換為三元組。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!cp data/onto.ttl .\n",
|
||||
"\n",
|
||||
"gedcom_dict = g.get_element_dictionary()\n",
|
||||
"individuals, marriages = {}, {}\n",
|
||||
"\n",
|
||||
"def term2id(el):\n",
|
||||
" return \"i\" + el.get_pointer().replace('@', '').lower()\n",
|
||||
"\n",
|
||||
"out = open(\"onto.ttl\",\"a\")\n",
|
||||
"\n",
|
||||
"for k, v in gedcom_dict.items():\n",
|
||||
" if isinstance(v,IndividualElement):\n",
|
||||
" children, siblings = set(), set()\n",
|
||||
" idx = term2id(v)\n",
|
||||
"\n",
|
||||
" title = v.get_name()[0] + \" \" + v.get_name()[1]\n",
|
||||
" title = title.replace('\"', '').replace('[', '').replace(']', '').replace('(', '').replace(')', '').strip()\n",
|
||||
"\n",
|
||||
" own_families = g.get_families(v, 'FAMS')\n",
|
||||
" for fam in own_families:\n",
|
||||
" children |= set(term2id(i) for i in g.get_family_members(fam, \"CHIL\"))\n",
|
||||
"\n",
|
||||
" parent_families = g.get_families(v, 'FAMC')\n",
|
||||
" if len(parent_families):\n",
|
||||
" for member in g.get_family_members(parent_families[0], \"CHIL\"): # NB adoptive families i.e len(parent_families)>1 are not considered (TODO?)\n",
|
||||
" if member.get_pointer() == v.get_pointer():\n",
|
||||
" continue\n",
|
||||
" siblings.add(term2id(member))\n",
|
||||
"\n",
|
||||
" if idx in individuals:\n",
|
||||
" children |= individuals[idx].get('children', set())\n",
|
||||
" siblings |= individuals[idx].get('siblings', set())\n",
|
||||
" individuals[idx] = {'sex': v.get_gender().lower(), 'children': children, 'siblings': siblings, 'title': title}\n",
|
||||
"\n",
|
||||
" elif isinstance(v,FamilyElement):\n",
|
||||
" wife, husb, children = None, None, set()\n",
|
||||
" children = set(term2id(i) for i in g.get_family_members(v, \"CHIL\"))\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" wife = g.get_family_members(v, \"WIFE\")[0]\n",
|
||||
" wife = term2id(wife)\n",
|
||||
" if wife in individuals: individuals[wife]['children'] |= children\n",
|
||||
" else: individuals[wife] = {'children': children}\n",
|
||||
" except IndexError: pass\n",
|
||||
" try:\n",
|
||||
" husb = g.get_family_members(v, \"HUSB\")[0]\n",
|
||||
" husb = term2id(husb)\n",
|
||||
" if husb in individuals: individuals[husb]['children'] |= children\n",
|
||||
" else: individuals[husb] = {'children': children}\n",
|
||||
" except IndexError: pass\n",
|
||||
"\n",
|
||||
" if wife and husb: marriages[wife + husb] = (term2id(v), wife, husb)\n",
|
||||
"\n",
|
||||
"for idx, val in individuals.items():\n",
|
||||
" added_terms = ''\n",
|
||||
" if val['sex'] == 'f':\n",
|
||||
" parent_predicate, sibl_predicate = \"isMotherOf\", \"isSisterOf\"\n",
|
||||
" else:\n",
|
||||
" parent_predicate, sibl_predicate = \"isFatherOf\", \"isBrotherOf\"\n",
|
||||
" if len(val['children']):\n",
|
||||
" added_terms += \" ;\\n fhkb:\" + parent_predicate + \" \" + \", \".join([\"fhkb:\" + i for i in val['children']])\n",
|
||||
" if len(val['siblings']):\n",
|
||||
" added_terms += \" ;\\n fhkb:\" + sibl_predicate + \" \" + \", \".join([\"fhkb:\" + i for i in val['siblings']])\n",
|
||||
" out.write(\"fhkb:%s a owl:NamedIndividual, owl:Thing%s ;\\n rdfs:label \\\"%s\\\" .\\n\" % (idx, added_terms, val['title']))\n",
|
||||
"\n",
|
||||
"for k, v in marriages.items():\n",
|
||||
" out.write(\"fhkb:%s a owl:NamedIndividual, owl:Thing ;\\n fhkb:hasFemalePartner fhkb:%s ;\\n fhkb:hasMalePartner fhkb:%s .\\n\" % v)\n",
|
||||
"\n",
|
||||
"out.write(\"[] a owl:AllDifferent ;\\n owl:distinctMembers (\")\n",
|
||||
"for idx in individuals.keys():\n",
|
||||
" out.write(\" fhkb:\" + idx)\n",
|
||||
"for k, v in marriages.items():\n",
|
||||
" out.write(\" fhkb:\" + v[0])\n",
|
||||
"out.write(\" ) .\")\n",
|
||||
"out.close()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" fhkb:hasFemalePartner fhkb:i34 ;\n",
|
||||
" fhkb:hasMalePartner fhkb:i33 .\n",
|
||||
"fhkb:i54 a owl:NamedIndividual, owl:Thing ;\n",
|
||||
" fhkb:hasFemalePartner fhkb:i36 ;\n",
|
||||
" fhkb:hasMalePartner fhkb:i35 .\n",
|
||||
"fhkb:i55 a owl:NamedIndividual, owl:Thing ;\n",
|
||||
" fhkb:hasFemalePartner fhkb:i38 ;\n",
|
||||
" fhkb:hasMalePartner fhkb:i37 .\n",
|
||||
"[] a owl:AllDifferent ;\n",
|
||||
" owl:distinctMembers ( fhkb:i0 fhkb:i1 fhkb:i2 fhkb:i3 fhkb:i4 fhkb:i5 fhkb:i6 fhkb:i7 fhkb:i8 fhkb:i9 fhkb:i10 fhkb:i11 fhkb:i12 fhkb:i13 fhkb:i14 fhkb:i15 fhkb:i16 fhkb:i17 fhkb:i18 fhkb:i19 fhkb:i20 fhkb:i21 fhkb:i22 fhkb:i23 fhkb:i24 fhkb:i25 fhkb:i26 fhkb:i27 fhkb:i28 fhkb:i29 fhkb:i30 fhkb:i31 fhkb:i32 fhkb:i33 fhkb:i34 fhkb:i35 fhkb:i36 fhkb:i37 fhkb:i38 fhkb:i39 fhkb:i40 fhkb:i41 fhkb:i42 fhkb:i43 fhkb:i44 fhkb:i45 fhkb:i46 fhkb:i47 fhkb:i48 fhkb:i49 fhkb:i50 fhkb:i51 fhkb:i52 fhkb:i53 fhkb:i54 fhkb:i55 ) ."
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!tail onto.ttl"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 進行推理\n",
|
||||
"\n",
|
||||
"現在我們希望能夠使用這個本體進行推理和查詢。我們將使用 [RDFLib](https://github.com/RDFLib)，這是一個用於以不同格式讀取 RDF 圖、查詢等的庫。\n",
|
||||
"\n",
|
||||
"對於邏輯推理，我們將使用 [OWL-RL](https://github.com/RDFLib/OWL-RL) 庫，該庫允許我們構建 RDF 圖的 **Closure**，也就是添加所有可以推導出的概念和關係。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Requirement already satisfied: rdflib in /home/rg/anaconda3/envs/ai4beg/lib/python3.11/site-packages (6.3.2)\n",
|
||||
"Requirement already satisfied: isodate<0.7.0,>=0.6.0 in /home/rg/anaconda3/envs/ai4beg/lib/python3.11/site-packages (from rdflib) (0.6.1)\n",
|
||||
"Requirement already satisfied: pyparsing<4,>=2.1.0 in /home/rg/anaconda3/envs/ai4beg/lib/python3.11/site-packages (from rdflib) (3.0.9)\n",
|
||||
"Requirement already satisfied: six in /home/rg/anaconda3/envs/ai4beg/lib/python3.11/site-packages (from isodate<0.7.0,>=0.6.0->rdflib) (1.16.0)\n",
|
||||
"Collecting git+https://github.com/RDFLib/OWL-RL.git\n",
|
||||
" Cloning https://github.com/RDFLib/OWL-RL.git to /tmp/pip-req-build-lbfzwi3m\n",
|
||||
" Running command git clone --filter=blob:none --quiet https://github.com/RDFLib/OWL-RL.git /tmp/pip-req-build-lbfzwi3m\n",
|
||||
" Resolved https://github.com/RDFLib/OWL-RL.git to commit a77e1791b88b54aace609bc6000aac14c7add4ff\n",
|
||||
" Preparing metadata (setup.py) ... \u001b[?25ldone\n",
|
||||
"\u001b[?25hRequirement already satisfied: rdflib>=6.0.2 in /home/rg/anaconda3/envs/ai4beg/lib/python3.11/site-packages (from owlrl==6.0.2) (6.3.2)\n",
|
||||
"Requirement already satisfied: isodate<0.7.0,>=0.6.0 in /home/rg/anaconda3/envs/ai4beg/lib/python3.11/site-packages (from rdflib>=6.0.2->owlrl==6.0.2) (0.6.1)\n",
|
||||
"Requirement already satisfied: pyparsing<4,>=2.1.0 in /home/rg/anaconda3/envs/ai4beg/lib/python3.11/site-packages (from rdflib>=6.0.2->owlrl==6.0.2) (3.0.9)\n",
|
||||
"Requirement already satisfied: six in /home/rg/anaconda3/envs/ai4beg/lib/python3.11/site-packages (from isodate<0.7.0,>=0.6.0->rdflib>=6.0.2->owlrl==6.0.2) (1.16.0)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!{sys.executable} -m pip install rdflib\n",
|
||||
"!{sys.executable} -m pip install git+https://github.com/RDFLib/OWL-RL.git"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"讓我們打開本體文件，看看它包含多少三元組：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Triplets found:669\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import rdflib\n",
|
||||
"from owlrl import DeductiveClosure, OWLRL_Extension\n",
|
||||
"\n",
|
||||
"g = rdflib.Graph()\n",
|
||||
"g.parse(\"onto.ttl\", format=\"turtle\")\n",
|
||||
"\n",
|
||||
"print(\"Triplets found:%d\" % len(g))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Triplets after inference:4246\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"DeductiveClosure(OWLRL_Extension).expand(g)\n",
|
||||
"print(\"Triplets after inference:%d\" % len(g))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 查詢親屬關係\n",
|
||||
"\n",
|
||||
"現在我們可以查詢圖表，查看人與人之間的不同關係。我們可以使用 **SPARQL** 語言搭配 `query` 方法。在我們的例子中，讓我們看看家譜中的所有 **叔叔**：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Fedor Alekseevich Romanov is uncle of Ekaterina Ivanovna Romanova\n",
|
||||
"Aleksandr I Pavlovich Romanov is uncle of Aleksandr II Nikolaevich Romanov\n",
|
||||
"Fedor Alekseevich Romanov is uncle of Anna Ivanovna Romanova\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"qres = g.query(\n",
|
||||
" \"\"\"SELECT DISTINCT ?aname ?bname\n",
|
||||
" WHERE {\n",
|
||||
" ?a fhkb:isUncleOf ?b .\n",
|
||||
" ?a rdfs:label ?aname .\n",
|
||||
" ?b rdfs:label ?bname .\n",
|
||||
" }\"\"\")\n",
|
||||
"\n",
|
||||
"for row in qres:\n",
|
||||
" print(\"%s is uncle of %s\" % row)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"隨意嘗試其他家庭關係。例如，你可以看看 `isAncestorOf` 關係，它遞歸地定義了一個人所有的祖先。\n",
|
||||
"\n",
|
||||
"最後，讓我們清理一下吧！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!rm onto.ttl"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免責聲明**： \n本文件使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們努力確保翻譯的準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解釋不承擔責任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.6",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.2"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "6537d5597320e27b6052b4377b8ff8bb",
|
||||
"translation_date": "2025-08-28T11:28:52+00:00",
|
||||
"source_file": "lessons/2-Symbolic/FamilyOntology.ipynb",
|
||||
"language_code": "mo"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,548 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"## Microsoft Concept Graph - 此 API 已不再可用，但請查看筆記本以了解概念\n",
|
||||
"\n",
|
||||
"[Microsoft Concept Graph](https://concept.research.microsoft.com/) 是一個從互聯網挖掘的龐大術語分類，包含概念之間的 `is-a` 關係。\n",
|
||||
"\n",
|
||||
"Context Graph 提供以下兩種形式：\n",
|
||||
" * 可下載的大型文本文件\n",
|
||||
" * REST API\n",
|
||||
"\n",
|
||||
"統計數據：\n",
|
||||
" * 5401933 個獨特概念\n",
|
||||
" * 12551613 個獨特實例\n",
|
||||
" * 87603947 個 `is-a` 關係\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 使用網路服務\n",
|
||||
"\n",
|
||||
"網路服務提供不同的呼叫方式，用於估算某個概念屬於不同群組的可能性。更多資訊請參考[這裡](https://concept.research.microsoft.com/Home/Api)。 \n",
|
||||
"以下是範例 URL 呼叫：`https://concept.research.microsoft.com/api/Concept/ScoreByProb?instance=microsoft&topK=10`\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'company': 0.6105356614382954,\n",
|
||||
" 'vendor': 0.08858636677518003,\n",
|
||||
" 'client': 0.048239124001183784,\n",
|
||||
" 'firm': 0.045476965571668145,\n",
|
||||
" 'large company': 0.043109401203511886,\n",
|
||||
" 'organization': 0.043010752688172046,\n",
|
||||
" 'corporation': 0.035908059583703265,\n",
|
||||
" 'brand': 0.03383644076156654,\n",
|
||||
" 'software company': 0.027522935779816515,\n",
|
||||
" 'technology company': 0.023774292196902438}"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import urllib\n",
|
||||
"import json\n",
|
||||
"import ssl\n",
|
||||
"\n",
|
||||
"def http(x):\n",
|
||||
" ssl._create_default_https_context = ssl._create_unverified_context\n",
|
||||
" response = urllib.request.urlopen(x)\n",
|
||||
" data = response.read()\n",
|
||||
" return data.decode('utf-8')\n",
|
||||
"\n",
|
||||
"def query(x):\n",
|
||||
" return json.loads(http(\"https://concept.research.microsoft.com/api/Concept/ScoreByProb?instance={}&topK=10\".format(urllib.parse.quote(x))))\n",
|
||||
"\n",
|
||||
"query('microsoft')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"讓我們嘗試使用父概念對新聞標題進行分類。要獲取新聞標題，我們將使用 [NewsApi.org](http://newsapi.org) 服務。您需要獲取自己的 API 金鑰才能使用該服務——請訪問網站並註冊免費的開發者計劃。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 20,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"newsapi_key = '<your API key here>'\n",
|
||||
"def get_news(country='us'):\n",
|
||||
" res = json.loads(http(\"https://newsapi.org/v2/top-headlines?country={0}&apiKey={1}\".format(country,newsapi_key)))\n",
|
||||
" return res['articles']\n",
|
||||
"\n",
|
||||
"all_titles = [x['title'] for x in get_news('us')+get_news('gb')]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 21,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['Covid-19 Live Updates: Vaccines and Boosters News - The New York Times',\n",
|
||||
" 'Ukrainians Flee Mariupol as Russian Forces Push to Take Port City - The Wall Street Journal',\n",
|
||||
" 'Bond Yields Jump, Stock Futures Rise After Powell Says Fed Is Ready to Be More Aggressive - The Wall Street Journal',\n",
|
||||
" 'Putin critic Alexei Navalny found guilty by Russian court - New York Post ',\n",
|
||||
" \"Supreme Court nominee Ketanji Brown Jackson will face questions at confirmation hearing's second day - CNN\",\n",
|
||||
" '2 teachers killed at Swedish high school, student arrested - ABC News',\n",
|
||||
" 'Clues to Covid-19’s Next Moves Come From Sewers - The Wall Street Journal',\n",
|
||||
" 'Republicans to roll dice by grilling Jackson over child-pornography sentencing decisions | TheHill - The Hill',\n",
|
||||
" '‘Clear sign’ Putin considering using chemical weapons in Ukraine, claims President Biden - The Independent',\n",
|
||||
" 'NASA confirms there are 5,000 planets outside our solar system - Daily Mail',\n",
|
||||
" \"US stocks whipsawed overnight after Fed Chair Powell's remarks - Fox Business\",\n",
|
||||
" \"'We've learned absolutely nothing': Tests could again be in short supply if Covid surges - POLITICO\",\n",
|
||||
" \"Duchess of Cambridge swaps khaki jungle gear for Vampire's Wife dress on Belize trip - Daily Mail\",\n",
|
||||
" 'China searches for victims, flight recorders after first plane crash in 12 years - Reuters',\n",
|
||||
" 'Second superyacht linked to Russian oligarch Abramovich docks in Turkey - Reuters',\n",
|
||||
" 'Live updates: Russia stops talks with Japan over sanctions - The Associated Press - en Español',\n",
|
||||
" 'Powers Remain and Threats Lurk as Women’s Sweet 16 Is Set - The New York Times',\n",
|
||||
" 'Webb Space Telescope Begins Multi-Instrument Alignment - SciTechDaily',\n",
|
||||
" \"UConn vs UCF - NCAA women's tournament second-round highlights - March Madness\",\n",
|
||||
" 'Bucking Republican Trend, Indiana Governor Vetoes Transgender Sports Bill - The New York Times',\n",
|
||||
" \"Maggie Fox dead: Coronation Street and Shameless actress dies after 'sudden accident' - Mirror Online - The Mirror\",\n",
|
||||
" 'China plane crash – live: Search for survivors continues as witness describes moment flight fell from sky - The Independent',\n",
|
||||
" 'Daniel Morgan murder: damning report condemns Met police - The Guardian',\n",
|
||||
" 'What to expect from Rishi Sunak’s Spring Statement - BBC.com',\n",
|
||||
" 'UK and Republic of Ireland in line to host Euro 2028 after no one else bids - The Guardian',\n",
|
||||
" \"Friends beg Vladimir Putin's 'lover' to persuade him to end Ukraine invasion - The Mirror\",\n",
|
||||
" 'Brass Eye’s outtakes show the brutal TV comedy was the tip of an iceberg - The Guardian',\n",
|
||||
" \"Vladimir Putin threatens civilians to break Mariupol's spirit - The Times\",\n",
|
||||
" 'Shell U-turn on Cambo oilfield would threaten green targets, say campaigners - The Guardian',\n",
|
||||
" 'St Helens dog attack: Girl aged 17 months killed at home - BBC',\n",
|
||||
" \"PlayStation to buy 'Assassin's Creed' veteran Jade Raymond's Haven Studios - NME\",\n",
|
||||
" '‘Clear sign’ Putin considering using chemical weapons in Ukraine, claims President Biden - The Independent',\n",
|
||||
" 'NASA confirms there are 5,000 planets outside our solar system - Daily Mail',\n",
|
||||
" 'Nintendo Switch finally has folders • Eurogamer.net - Eurogamer.net',\n",
|
||||
" 'FA to “find a solution” as Liverpool fan group blasts “shambolic” Wembley travel - This Is Anfield',\n",
|
||||
" 'Manchester United transfer news LIVE Erik ten Hag latest and Man Utd manager updates - Manchester Evening News',\n",
|
||||
" 'Inflation raises cost of UK government borrowing in February; crude oil up again – business live - The Guardian',\n",
|
||||
" 'Alexei Navalny: Kremlin critic found guilty of large-scale fraud and contempt of court by Russian court - Sky News',\n",
|
||||
" \"UK prepares to nationalize Russia natural gas giant Gazprom's retail unit - Business Insider\",\n",
|
||||
" 'Zaghari-Ratcliffe: Hunt calls for inquiry into delay over Iran debt payment - The Guardian']"
|
||||
]
|
||||
},
|
||||
"execution_count": 21,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"all_titles"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"首先，我們希望能夠從新聞標題中提取名詞。我們將使用 `TextBlob` 庫來完成這項工作，該庫簡化了許多像這樣的典型自然語言處理任務。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Requirement already satisfied: textblob in c:\\winapp\\miniconda3\\lib\\site-packages (0.17.1)\n",
|
||||
"Requirement already satisfied: nltk>=3.1 in c:\\winapp\\miniconda3\\lib\\site-packages (from textblob) (3.5)\n",
|
||||
"Requirement already satisfied: joblib in c:\\winapp\\miniconda3\\lib\\site-packages (from nltk>=3.1->textblob) (1.0.1)\n",
|
||||
"Requirement already satisfied: regex in c:\\winapp\\miniconda3\\lib\\site-packages (from nltk>=3.1->textblob) (2021.11.10)\n",
|
||||
"Requirement already satisfied: tqdm in c:\\winapp\\miniconda3\\lib\\site-packages (from nltk>=3.1->textblob) (4.61.2)\n",
|
||||
"Requirement already satisfied: click in c:\\winapp\\miniconda3\\lib\\site-packages (from nltk>=3.1->textblob) (8.0.3)\n",
|
||||
"Requirement already satisfied: colorama in c:\\winapp\\miniconda3\\lib\\site-packages (from click->nltk>=3.1->textblob) (0.4.4)\n",
|
||||
"Finished.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[nltk_data] Downloading package brown to\n",
|
||||
"[nltk_data] C:\\Users\\dmitryso\\AppData\\Roaming\\nltk_data...\n",
|
||||
"[nltk_data] Package brown is already up-to-date!\n",
|
||||
"[nltk_data] Downloading package punkt to\n",
|
||||
"[nltk_data] C:\\Users\\dmitryso\\AppData\\Roaming\\nltk_data...\n",
|
||||
"[nltk_data] Package punkt is already up-to-date!\n",
|
||||
"[nltk_data] Downloading package wordnet to\n",
|
||||
"[nltk_data] C:\\Users\\dmitryso\\AppData\\Roaming\\nltk_data...\n",
|
||||
"[nltk_data] Package wordnet is already up-to-date!\n",
|
||||
"[nltk_data] Downloading package averaged_perceptron_tagger to\n",
|
||||
"[nltk_data] C:\\Users\\dmitryso\\AppData\\Roaming\\nltk_data...\n",
|
||||
"[nltk_data] Package averaged_perceptron_tagger is already up-to-\n",
|
||||
"[nltk_data] date!\n",
|
||||
"[nltk_data] Downloading package conll2000 to\n",
|
||||
"[nltk_data] C:\\Users\\dmitryso\\AppData\\Roaming\\nltk_data...\n",
|
||||
"[nltk_data] Package conll2000 is already up-to-date!\n",
|
||||
"[nltk_data] Downloading package movie_reviews to\n",
|
||||
"[nltk_data] C:\\Users\\dmitryso\\AppData\\Roaming\\nltk_data...\n",
|
||||
"[nltk_data] Package movie_reviews is already up-to-date!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"!{sys.executable} -m pip install textblob\n",
|
||||
"!{sys.executable} -m textblob.download_corpora\n",
|
||||
"from textblob import TextBlob"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 22,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'covid-19 live updates': 1,\n",
|
||||
" 'vaccines': 1,\n",
|
||||
" 'boosters': 1,\n",
|
||||
" 'york': 4,\n",
|
||||
" 'ukrainians flee mariupol': 1,\n",
|
||||
" 'forces push': 1,\n",
|
||||
" 'port city': 1,\n",
|
||||
" 'wall street journal': 3,\n",
|
||||
" 'bond yields': 1,\n",
|
||||
" 'futures rise': 1,\n",
|
||||
" 'powell says fed': 1,\n",
|
||||
" 'ready': 1,\n",
|
||||
" 'be': 1,\n",
|
||||
" 'aggressive': 1,\n",
|
||||
" 'putin': 3,\n",
|
||||
" 'alexei navalny': 2,\n",
|
||||
" 'russian': 2,\n",
|
||||
" 'supreme court nominee': 1,\n",
|
||||
" 'ketanji brown jackson': 1,\n",
|
||||
" \"confirmation hearing 's\": 1,\n",
|
||||
" 'cnn': 1,\n",
|
||||
" 'swedish': 1,\n",
|
||||
" 'high school': 1,\n",
|
||||
" 'abc': 1,\n",
|
||||
" 'clues': 1,\n",
|
||||
" 'covid-19': 1,\n",
|
||||
" '’ s': 2,\n",
|
||||
" 'moves': 1,\n",
|
||||
" 'sewers': 1,\n",
|
||||
" 'roll dice': 1,\n",
|
||||
" 'jackson': 1,\n",
|
||||
" 'decisions |': 1,\n",
|
||||
" 'thehill': 1,\n",
|
||||
" 'clear': 2,\n",
|
||||
" 'chemical weapons': 2,\n",
|
||||
" 'ukraine': 3,\n",
|
||||
" 'claims president': 2,\n",
|
||||
" 'biden': 2,\n",
|
||||
" 'nasa': 2,\n",
|
||||
" 'solar system': 2,\n",
|
||||
" 'daily mail': 3,\n",
|
||||
" 'us stocks': 1,\n",
|
||||
" 'fed chair powell': 1,\n",
|
||||
" \"'s remarks\": 1,\n",
|
||||
" 'fox': 1,\n",
|
||||
" \"'we 've\": 1,\n",
|
||||
" 'tests': 1,\n",
|
||||
" 'covid': 1,\n",
|
||||
" 'politico': 1,\n",
|
||||
" 'duchess': 1,\n",
|
||||
" 'cambridge': 1,\n",
|
||||
" 'swaps khaki jungle gear': 1,\n",
|
||||
" 'vampire': 1,\n",
|
||||
" 'wife': 1,\n",
|
||||
" 'belize': 1,\n",
|
||||
" 'china': 2,\n",
|
||||
" 'flight recorders': 1,\n",
|
||||
" 'plane crash': 1,\n",
|
||||
" 'reuters': 2,\n",
|
||||
" 'russian oligarch': 1,\n",
|
||||
" 'abramovich': 1,\n",
|
||||
" 'live': 1,\n",
|
||||
" 'russia': 2,\n",
|
||||
" 'stops talks': 1,\n",
|
||||
" 'japan': 1,\n",
|
||||
" 'español': 1,\n",
|
||||
" 'powers remain': 1,\n",
|
||||
" 'threats lurk': 1,\n",
|
||||
" 'set': 1,\n",
|
||||
" 'webb': 1,\n",
|
||||
" 'telescope begins multi-instrument alignment': 1,\n",
|
||||
" 'scitechdaily': 1,\n",
|
||||
" 'uconn': 1,\n",
|
||||
" 'ucf': 1,\n",
|
||||
" 'ncaa': 1,\n",
|
||||
" \"women 's tournament second-round highlights\": 1,\n",
|
||||
" 'march madness': 1,\n",
|
||||
" 'bucking republican trend': 1,\n",
|
||||
" 'indiana': 1,\n",
|
||||
" 'vetoes transgender': 1,\n",
|
||||
" 'bill': 1,\n",
|
||||
" 'maggie fox': 1,\n",
|
||||
" 'coronation': 1,\n",
|
||||
" 'shameless': 1,\n",
|
||||
" \"'sudden accident\": 1,\n",
|
||||
" 'mirror online': 1,\n",
|
||||
" 'mirror': 2,\n",
|
||||
" 'plane crash –': 1,\n",
|
||||
" 'search': 1,\n",
|
||||
" 'moment flight': 1,\n",
|
||||
" 'daniel morgan': 1,\n",
|
||||
" 'report condemns': 1,\n",
|
||||
" 'met': 1,\n",
|
||||
" 'guardian': 6,\n",
|
||||
" 'rishi sunak': 1,\n",
|
||||
" '’ s spring': 1,\n",
|
||||
" 'statement': 1,\n",
|
||||
" 'bbc.com': 1,\n",
|
||||
" 'uk': 3,\n",
|
||||
" 'ireland': 1,\n",
|
||||
" 'euro': 1,\n",
|
||||
" 'vladimir putin': 2,\n",
|
||||
" \"'s 'lover\": 1,\n",
|
||||
" 'brass eye': 1,\n",
|
||||
" '’ s outtakes': 1,\n",
|
||||
" 'brutal tv comedy': 1,\n",
|
||||
" 'threatens civilians': 1,\n",
|
||||
" 'mariupol': 1,\n",
|
||||
" \"'s spirit\": 1,\n",
|
||||
" 'shell u-turn': 1,\n",
|
||||
" 'cambo': 1,\n",
|
||||
" 'green targets': 1,\n",
|
||||
" 'st helens': 1,\n",
|
||||
" 'dog attack': 1,\n",
|
||||
" 'girl': 1,\n",
|
||||
" 'bbc': 1,\n",
|
||||
" 'playstation': 1,\n",
|
||||
" \"'assassin 's\": 1,\n",
|
||||
" 'creed': 1,\n",
|
||||
" 'jade raymond': 1,\n",
|
||||
" 'haven studios': 1,\n",
|
||||
" 'nme': 1,\n",
|
||||
" 'nintendo switch': 1,\n",
|
||||
" 'folders •': 1,\n",
|
||||
" 'eurogamer.net': 2,\n",
|
||||
" 'fa': 1,\n",
|
||||
" 'solution ”': 1,\n",
|
||||
" 'liverpool': 1,\n",
|
||||
" 'fan group blasts “ shambolic ”': 1,\n",
|
||||
" 'wembley': 1,\n",
|
||||
" 'anfield': 1,\n",
|
||||
" 'manchester': 1,\n",
|
||||
" 'live erik': 1,\n",
|
||||
" 'hag': 1,\n",
|
||||
" 'utd': 1,\n",
|
||||
" 'manager updates': 1,\n",
|
||||
" 'manchester evening': 1,\n",
|
||||
" 'inflation': 1,\n",
|
||||
" 'government borrowing': 1,\n",
|
||||
" 'february': 1,\n",
|
||||
" 'crude oil': 1,\n",
|
||||
" '– business': 1,\n",
|
||||
" 'kremlin': 1,\n",
|
||||
" 'large-scale fraud': 1,\n",
|
||||
" 'sky': 1,\n",
|
||||
" 'natural gas': 1,\n",
|
||||
" 'gazprom': 1,\n",
|
||||
" 'retail unit': 1,\n",
|
||||
" 'insider': 1,\n",
|
||||
" 'zaghari-ratcliffe': 1,\n",
|
||||
" 'hunt': 1,\n",
|
||||
" 'iran': 1,\n",
|
||||
" 'debt payment': 1}"
|
||||
]
|
||||
},
|
||||
"execution_count": 22,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"w = {}\n",
|
||||
"for x in all_titles:\n",
|
||||
" for n in TextBlob(x).noun_phrases:\n",
|
||||
" if n in w:\n",
|
||||
" w[n].append(x)\n",
|
||||
" else:\n",
|
||||
" w[n]=[x]\n",
|
||||
"{ x:len(w[x]) for x in w.keys()}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我們可以看到名詞並未給我們大的主題群組。讓我們用從概念圖中獲得的更一般的術語來替代名詞。這將需要一些時間，因為我們正在為每個名詞短語進行 REST 調用。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 23,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"w = {}\n",
|
||||
"for x in all_titles:\n",
|
||||
" for noun in TextBlob(x).noun_phrases:\n",
|
||||
" terms = query(noun.replace(' ','%20'))\n",
|
||||
" for term in [u for u in terms.keys() if terms[u]>0.1]:\n",
|
||||
" if term in w:\n",
|
||||
" w[term].append(x)\n",
|
||||
" else:\n",
|
||||
" w[term]=[x]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 24,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'city': 9,\n",
|
||||
" 'brand': 4,\n",
|
||||
" 'place': 9,\n",
|
||||
" 'town': 4,\n",
|
||||
" 'factor': 4,\n",
|
||||
" 'film': 4,\n",
|
||||
" 'nation': 11,\n",
|
||||
" 'state': 5,\n",
|
||||
" 'person': 4,\n",
|
||||
" 'organization': 5,\n",
|
||||
" 'publication': 10,\n",
|
||||
" 'market': 5,\n",
|
||||
" 'economy': 4,\n",
|
||||
" 'company': 6,\n",
|
||||
" 'newspaper': 6,\n",
|
||||
" 'relationship': 6}"
|
||||
]
|
||||
},
|
||||
"execution_count": 24,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"{ x:len(w[x]) for x in w.keys() if len(w[x])>3}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 27,
|
||||
"metadata": {
|
||||
"trusted": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"ECONOMY:\n",
|
||||
"China searches for victims, flight recorders after first plane crash in 12 years - Reuters\n",
|
||||
"Live updates: Russia stops talks with Japan over sanctions - The Associated Press - en Español\n",
|
||||
"China plane crash – live: Search for survivors continues as witness describes moment flight fell from sky - The Independent\n",
|
||||
"UK prepares to nationalize Russia natural gas giant Gazprom's retail unit - Business Insider\n",
|
||||
"\n",
|
||||
"NATION:\n",
|
||||
"‘Clear sign’ Putin considering using chemical weapons in Ukraine, claims President Biden - The Independent\n",
|
||||
"Duchess of Cambridge swaps khaki jungle gear for Vampire's Wife dress on Belize trip - Daily Mail\n",
|
||||
"China searches for victims, flight recorders after first plane crash in 12 years - Reuters\n",
|
||||
"Live updates: Russia stops talks with Japan over sanctions - The Associated Press - en Español\n",
|
||||
"Live updates: Russia stops talks with Japan over sanctions - The Associated Press - en Español\n",
|
||||
"China plane crash – live: Search for survivors continues as witness describes moment flight fell from sky - The Independent\n",
|
||||
"UK and Republic of Ireland in line to host Euro 2028 after no one else bids - The Guardian\n",
|
||||
"Friends beg Vladimir Putin's 'lover' to persuade him to end Ukraine invasion - The Mirror\n",
|
||||
"‘Clear sign’ Putin considering using chemical weapons in Ukraine, claims President Biden - The Independent\n",
|
||||
"UK prepares to nationalize Russia natural gas giant Gazprom's retail unit - Business Insider\n",
|
||||
"Zaghari-Ratcliffe: Hunt calls for inquiry into delay over Iran debt payment - The Guardian\n",
|
||||
"\n",
|
||||
"PERSON:\n",
|
||||
"‘Clear sign’ Putin considering using chemical weapons in Ukraine, claims President Biden - The Independent\n",
|
||||
"Duchess of Cambridge swaps khaki jungle gear for Vampire's Wife dress on Belize trip - Daily Mail\n",
|
||||
"Second superyacht linked to Russian oligarch Abramovich docks in Turkey - Reuters\n",
|
||||
"‘Clear sign’ Putin considering using chemical weapons in Ukraine, claims President Biden - The Independent\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print('\\nECONOMY:\\n'+'\\n'.join(w['economy']))\n",
|
||||
"print('\\nNATION:\\n'+'\\n'.join(w['nation']))\n",
|
||||
"print('\\nPERSON:\\n'+'\\n'.join(w['person']))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免責聲明**： \n本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們努力確保準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵資訊，建議使用專業人工翻譯。我們對因使用此翻譯而產生的任何誤解或錯誤解釋不承擔責任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.7.4 64-bit (conda)",
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5"
|
||||
}
|
||||
},
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.5"
|
||||
},
|
||||
"coopTranslator": {
|
||||
"original_hash": "7b3f1fc049371bcf8649ac9fefdf0413",
|
||||
"translation_date": "2025-10-03T18:51:30+00:00",
|
||||
"source_file": "lessons/2-Symbolic/MSConceptGraph.ipynb",
|
||||
"language_code": "mo"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,247 @@
|
|||
# 知識表示與專家系統
|
||||
|
||||
|
||||
|
||||
> 手繪筆記由 [Tomomi Imura](https://twitter.com/girlie_mac) 製作
|
||||
|
||||
人工智能的追求基於對知識的探索，旨在像人類一樣理解世界。但你該如何著手呢？
|
||||
|
||||
## [課前測驗](https://ff-quizzes.netlify.app/en/ai/quiz/3)
|
||||
|
||||
在AI的早期階段，自上而下的創建智能系統方法（前一課討論過）很受歡迎。這個想法是從人們身上提取知識，轉換成機器可讀的形式，然後用以自動解決問題。這種方法基於兩個重要的理念：
|
||||
|
||||
* 知識表示
|
||||
* 推理
|
||||
|
||||
## 知識表示
|
||||
|
||||
符號AI中的一個重要概念是**知識**。區分知識與*資訊*或*資料*很重要。例如，人們常說書本包含知識，因為閱讀書本可成為專家。然而，書本裡實際包含的是稱為*資料*的東西，通過閱讀書本並整合這些資料進入我們的世界模型，我們將資料轉變成知識。
|
||||
|
||||
> ✅ **知識**是存在於我們腦海中，代表我們對世界理解的東西。它是透過一個積極的**學習**過程獲得，將我們接收到的資訊片段整合進我們對世界的主動模型中。
|
||||
|
||||
我們通常不會嚴格定義知識，而是通過[DIKW金字塔](https://en.wikipedia.org/wiki/DIKW_pyramid)將它與其他相關概念對應，其包含以下概念：
|
||||
|
||||
* **資料**是以物理媒介表示的東西，如書寫文本或口語詞彙。資料獨立於人類存在，且可在人與人之間傳遞。
|
||||
* **資訊**是在我們腦中如何解讀資料。例如，當聽到「電腦」一詞時，我們對它有一定理解。
|
||||
* **知識**是被整合到我們的世界模型中的資訊。例如，當我們知道電腦是什麼，我們開始對它如何運作、價格多少、用途等有想法。這些相互關聯的概念網絡形成了我們的知識。
|
||||
* **智慧**是我們對世界理解的更高一層，代表*元知識*，例如關於何時及如何使用知識的一些觀念。
|
||||
|
||||
<img src="../../../../translated_images/zh-MO/DIKW_Pyramid.94126f7d2bd8db5b.webp" width="30%"/>
|
||||
|
||||
*圖片來自 [維基百科](https://commons.wikimedia.org/w/index.php?curid=37705247)，作者 Longlivetheux - 自作，CC BY-SA 4.0*
|
||||
|
||||
因此，**知識表示**問題是找到有效的方式，將知識以資料形式表示在電腦內，讓其可自動使用。這可看作是一個光譜：
|
||||
|
||||
|
||||
|
||||
> 圖片由 [Dmitry Soshnikov](http://soshnikov.com) 提供
|
||||
|
||||
* 左側是電腦可有效使用的非常簡單的知識表示類型。最簡單的是算法表示，知識由電腦程式代表。但這不是呈現知識的最佳方式，因為它不夠靈活。存在於我們腦中的知識常常是非算法性的。
|
||||
* 右側是自然文字表示。它最強大，但無法用於自動推理。
|
||||
|
||||
> ✅ 想一下你如何在腦中表示知識並轉換成筆記？有沒有特定的格式能幫助你記憶？
|
||||
|
||||
## 電腦知識表示的分類
|
||||
|
||||
我們可將不同的電腦知識表示方法分類如下：
|
||||
|
||||
* **網路表示**基於我們腦中存在的相互關聯概念網絡。我們可嘗試在電腦中重現這些網絡，作為圖形表示——所謂的**語義網絡**。
|
||||
|
||||
1. **物件-屬性-值三元組**或**屬性-值對**。由於圖形可在電腦中表示為節點與邊列表，我們可用三元組列表表示語義網絡，其中包含物件、屬性、值。例如，我們構建以下關於程式語言的三元組：
|
||||
|
||||
物件 | 屬性 | 值
|
||||
-------|-----------|------
|
||||
Python | 是 | 非型別語言
|
||||
Python | 發明者 | Guido van Rossum
|
||||
Python | 區塊語法 | 縮排
|
||||
非型別語言 | 沒有 | 型別定義
|
||||
|
||||
> ✅ 想想三元組如何用於表示其他類型的知識。
|
||||
|
||||
2. **層次式表示**強調我們經常在腦中建立物件層次結構。例如，我們知道金絲雀是鳥，所有鳥都有翅膀，我們也知道金絲雀通常的顏色及飛行速度。
|
||||
|
||||
- **框架表示**基於將每個物件或物件類別表示為包含**槽位**的**框架**。槽位可有預設值、值限制或可調用的程序以獲得槽位值。所有框架形成的層次結構類似物件導向程式語言中的物件層次結構。
|
||||
- **劇本**是特殊類型的框架，用於表示可隨時間展開的複雜情境。
|
||||
|
||||
**Python**
|
||||
|
||||
槽位 | 值 | 預設值 | 範圍 |
|
||||
-----|-------|---------------|----------|
|
||||
名稱 | Python | | |
|
||||
是-一個 | 非型別語言 | | |
|
||||
變數寫法 | | 駝峰式 | |
|
||||
程式長度 | | | 5-5000 行 |
|
||||
區塊語法 | 縮排 | | |
|
||||
|
||||
3. **程序式表示**基於以可在條件觸發時執行的一系列動作表示知識。
|
||||
- 生產規則是if-then語句，允許我們推斷結論。例如，一位醫生可能有規則：**如果**患者有高燒**或**血液檢驗中C反應蛋白偏高，**那麼**他有發炎情況。一旦符合條件之一，就能推論發炎，並用於後續推理。
|
||||
- 算法可視為另一種程序式表示，儘管它們幾乎不直接用於基於知識的系統。
|
||||
|
||||
4. **邏輯**最初由亞里斯多德提出，作為代表普遍人類知識的方法。
|
||||
- 謂詞邏輯作為數學理論過於豐富而難以計算，因此通常使用其子集，如Prolog中使用的Horn子句。
|
||||
- 描述邏輯屬於一組邏輯系統，用於表示與推理物件層次結構的分散式知識表示，如*語義網路*。
|
||||
|
||||
## 專家系統
|
||||
|
||||
符號AI的早期成功之一是所謂的**專家系統**——設計用於充當有限問題領域專家的計算機系統。它基於從一個或多個人類專家提取的**知識庫**，並包含執行推理的**推理引擎**。
|
||||
|
||||
 | 
|
||||
---------------------------------------------|------------------------------------------------
|
||||
簡化的人類神經系統結構 | 基於知識系統的架構
|
||||
|
||||
專家系統是按照人類推理系統建造的，該系統包含**短期記憶**與**長期記憶**。類似地，在基於知識的系統中，我們區分以下組件：
|
||||
|
||||
* **問題記憶**：包含當前解決問題的知識，例如果病人體溫或血壓，是否有發炎等。這類知識又稱為**靜態知識**，因為它是表示我們目前對問題了解的快照——即所謂的*問題狀態*。
|
||||
* **知識庫**：代表有關問題領域的長期知識。它是從人類專家手動提取的，且在諮詢期間不會改變。由於可引導我們從一個問題狀態轉移到另一狀態，故亦稱為**動態知識**。
|
||||
* **推理引擎**：調度整個搜索問題狀態空間的過程，在必要時向用戶提問。並負責找出應用於每個狀態的正確規則。
|
||||
|
||||
舉例來說，考慮以下透過動物的物理特徵判定動物的專家系統：
|
||||
|
||||
|
||||
|
||||
> 圖片由 [Dmitry Soshnikov](http://soshnikov.com) 提供
|
||||
|
||||
此圖被稱為**AND-OR樹**，是生產規則集合的圖形表述。繪製樹狀圖在從專家處提取知識時非常有用。在電腦中表示知識，則更方便使用規則：
|
||||
|
||||
```
|
||||
IF the animal eats meat
|
||||
OR (animal has sharp teeth
|
||||
AND animal has claws
|
||||
AND animal has forward-looking eyes
|
||||
)
|
||||
THEN the animal is a carnivore
|
||||
```
|
||||
|
||||
你會看到，每個規則的左側條件與動作本質上是物件-屬性-值（OAV）三元組。**工作記憶**包含與當前解決問題相關的OAV三元組集合。**規則引擎**尋找條件符合的規則並執行它們，將新的三元組加入工作記憶。
|
||||
|
||||
> ✅ 自己擬定一棵你感興趣主題的AND-OR樹吧！
|
||||
|
||||
### 正向與反向推理
|
||||
|
||||
上述過程稱為**正向推理**。它從工作記憶中可用的初始問題數據開始，然後執行以下推理循環：
|
||||
|
||||
1. 若工作記憶中有目標屬性，停止並輸出結果
|
||||
2. 搜尋條件當前被滿足的所有規則——獲得**衝突集**規則
|
||||
3. 執行**衝突解決**——從衝突集選擇一條執行規則。可能的衝突解決策略：
|
||||
- 選擇知識庫中第一個合適規則
|
||||
- 隨機選擇一條規則
|
||||
- 選擇*較特定*的規則，即「左側」條件最多條件被滿足的規則
|
||||
4. 應用所選規則並將新知識插入問題狀態
|
||||
5. 從第1步重複
|
||||
|
||||
然而，有時我們想從問題的空白知識開始，通過提問推理以得出結論。例如醫學診斷時，我們通常不會在診斷前事先做所有醫學檢查，而是根據需要決定做哪些檢查。
|
||||
|
||||
這個流程可以用**反向推理**建模。它由**目標**驅動——我們想找到的屬性值：
|
||||
|
||||
1. 選擇所有可給定目標值的規則（即目標在「右側」）——形成衝突集
|
||||
2. 若該屬性無規則或規則要求向使用者提問該值，則詢問使用者，否則：
|
||||
3. 使用衝突解決策略選擇一條規則作為*假設*，將嘗試證明它
|
||||
4. 遞迴地重複此過程，嘗試證明該規則左側（條件）的所有屬性目標
|
||||
5. 若過程中任一步失敗，返回步驟3選用另一規則
|
||||
|
||||
> ✅ 正向推理在何種情況較適合？反向推理又如何？
|
||||
|
||||
### 實作專家系統
|
||||
|
||||
專家系統可使用不同工具實現：
|
||||
|
||||
* 直接用高階程式語言編程。但這不是理想方法，因為基於知識系統的最大優點是知識和推理分離，而理想上問題領域專家應能不懂推理細節便撰寫規則。
|
||||
* 使用**專家系統殼層**，即專門設計用知識表示語言填充知識的系統。
|
||||
|
||||
## ✍️ 練習：動物推理
|
||||
|
||||
見[Animals.ipynb](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/2-Symbolic/Animals.ipynb)，範例展示如何實作正向與反向推理專家系統。
|
||||
|
||||
> **注意**：此範例較簡單，僅展示專家系統的基本樣貌。當你開始打造此系統時，只有達到一定數量的規則（約200條以上）時，才會看到一些*智能*行為。某階段規則過於複雜，難以全盤掌握，你會開始質疑系統為何做出某些決策。然而，基於知識系統的重要特性是，你總能*解釋*任何決定是如何做出的。
|
||||
|
||||
## 本體論與語義網路
|
||||
|
||||
20世紀末期曾有一項倡議，利用知識表示為網際網路資源加註解，使其可以找到符合非常具體查詢的資源。此倡議稱為**語義網路**，並依賴以下幾個概念：
|
||||
|
||||
- 基於**[描述邏輯](https://en.wikipedia.org/wiki/Description_logic)**(DL)的特殊知識表示。它類似框架知識表示，因為建立了帶屬性的物件層次結構，但擁有正式的邏輯語義與推理。描述邏輯有整個家族，平衡了表達力與推理算法複雜度。
|
||||
- 分散式知識表示，所有概念以全球URI識別，使得可建立跨網路的知識層次結構。
|
||||
- 一組基於 XML 的知識描述語言家族：RDF（資源描述框架）、RDFS（RDF 架構）、OWL（本體網路語言）。
|
||||
|
||||
語意網的一個核心概念是**本體**。它指的是使用某種形式化知識表示對問題域的明確規範。最簡單的本體可以只是問題域中物件的階層結構，但更複雜的本體將包括可用於推理的規則。
|
||||
|
||||
在語意網中，所有表示都是基於三元組。每個物件和每個關係都由 URI 唯一標識。例如，如果我們想表示這個 AI 課程是由 Dmitry Soshnikov 於 2022 年 1 月 1 日開發的——這裡是我們可以使用的三元組：
|
||||
|
||||
<img src="../../../../translated_images/zh-MO/triplet.4b9b332587593298.webp" width="30%"/>
|
||||
|
||||
```
|
||||
http://github.com/microsoft/ai-for-beginners http://www.example.com/terms/creation-date “Jan 1, 2022”
|
||||
http://github.com/microsoft/ai-for-beginners http://purl.org/dc/elements/1.1/creator http://soshnikov.com
|
||||
```
|
||||
|
||||
> ✅ 這裡 `http://www.example.com/terms/creation-date` 和 `http://purl.org/dc/elements/1.1/creator` 是一些眾所周知且被普遍接受的 URI，用於表達*創建者*和*創建日期*的概念。
|
||||
|
||||
在更複雜的情況下，如果我們想定義一個創建者列表，可以使用 RDF 定義的一些資料結構。
|
||||
|
||||
<img src="../../../../translated_images/zh-MO/triplet-complex.32094972c7b4441b.webp" width="40%"/>
|
||||
|
||||
> 上方圖示由 [Dmitry Soshnikov](http://soshnikov.com) 製作
|
||||
|
||||
語意網的建設進展在某種程度上因搜尋引擎和自然語言處理技術的成功而放緩，這些技術可以從文本中提取結構化數據。然而，在某些領域仍有大量努力來維護本體和知識庫。值得注意的幾個項目：
|
||||
|
||||
* [WikiData](https://wikidata.org/) 是一個與維基百科關聯的機器可讀知識庫集合。大部分數據來自維基百科的 *資訊框*，即維基百科頁面中的結構化內容片段。你可以使用 SPARQL，一種語意網專用查詢語言，來[查詢](https://query.wikidata.org/) wikidata。這裡有一個示例查詢，顯示人類中最流行的眼睛顏色：
|
||||
|
||||
```sparql
|
||||
#defaultView:BubbleChart
|
||||
SELECT ?eyeColorLabel (COUNT(?human) AS ?count)
|
||||
WHERE
|
||||
{
|
||||
?human wdt:P31 wd:Q5. # human instance-of homo sapiens
|
||||
?human wdt:P1340 ?eyeColor. # human eye-color ?eyeColor
|
||||
SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
|
||||
}
|
||||
GROUP BY ?eyeColorLabel
|
||||
```
|
||||
|
||||
* [DBpedia](https://www.dbpedia.org/) 是另一個類似 WikiData 的項目。
|
||||
|
||||
> ✅ 如果你想嘗試自己建立本體，或打開現有的本體，有一個很棒的視覺本體編輯器叫做 [Protégé](https://protege.stanford.edu/)。你可以下載使用，也可以線上使用。
|
||||
|
||||
<img src="../../../../translated_images/zh-MO/protege.274177ceeac13b38.webp" width="70%"/>
|
||||
|
||||
*Web Protégé 編輯器開啟了羅曼諾夫家族本體。截圖由 Dmitry Soshnikov 提供*
|
||||
|
||||
## ✍️ 練習：家族本體
|
||||
|
||||
參考 [FamilyOntology.ipynb](https://github.com/Ezana135/AI-For-Beginners/blob/main/lessons/2-Symbolic/FamilyOntology.ipynb) 了解如何使用語意網技術來推理家族關係。我們將採用以常見的 GEDCOM 格式表示的家譜和家族關係本體，構建給定個人集的所有家族關係圖譜。
|
||||
|
||||
## 微軟概念圖
|
||||
|
||||
在大多數情況下，本體是由人工精心創建的。然而，也可以從非結構化資料中**挖掘**本體，例如從自然語言文本中挖掘。
|
||||
|
||||
微軟研究院進行過這樣的嘗試，並產生了[微軟概念圖](https://blogs.microsoft.com/ai/microsoft-researchers-release-graph-that-helps-machines-conceptualize/?WT.mc_id=academic-77998-cacaste)。
|
||||
|
||||
這是一個使用「is-a」繼承關係將實體分組的大型集合。它允許回答「什麼是微軟？」這樣的問題——答案會是「以 0.87 的概率是一家公司，以 0.75 的概率是一個品牌」等。
|
||||
|
||||
該圖可通過 REST API 獲取，或下載一個列出所有實體對的大型文本文件。
|
||||
|
||||
## ✍️ 練習：概念圖
|
||||
|
||||
嘗試使用 [MSConceptGraph.ipynb](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/2-Symbolic/MSConceptGraph.ipynb) 筆記本，看看如何使用微軟概念圖將新聞文章分組到幾個類別中。
|
||||
|
||||
## 結論
|
||||
|
||||
現在，AI 常被認為是*機器學習*或*神經網絡*的同義詞。然而，人類也表現出明確的推理能力，這是目前神經網絡尚未處理的。在現實世界的專案中，明確推理仍用於執行需要解釋或能以可控方式修改系統行為的任務。
|
||||
|
||||
## 🚀 挑戰
|
||||
|
||||
在本課程相關的家族本體筆記本中，有機會嘗試其他家族關係。嘗試發掘家譜中人物之間的新連結。
|
||||
|
||||
## [課後測驗](https://ff-quizzes.netlify.app/en/ai/quiz/4)
|
||||
|
||||
## 複習與自學
|
||||
|
||||
在網路上做些調查，發掘人類嘗試量化和編碼知識的領域。看看布魯姆的分類法，回顧歷史，了解人類如何試圖理解世界。探索林奈創建生物分類法的工作，觀察德米特里·門捷列夫如何創造化學元素的描述和分組方式。你還可以找到哪些有趣的例子？
|
||||
|
||||
**作業**: [建立一個本體](assignment.md)
|
||||
|
||||
---
|
||||
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER START -->
|
||||
**免責聲明**：
|
||||
本文件係使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們力求準確，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威資料。對於重要資訊，建議採用專業人工翻譯。本公司對因使用此翻譯所產生的任何誤解或誤釋不承擔任何責任。
|
||||
<!-- CO-OP TRANSLATOR DISCLAIMER END -->
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# 建立本體
|
||||
|
||||
建立知識庫的核心在於對某個主題的事實進行分類並建構模型。選擇一個主題——例如一個人、一個地方或一件事物——然後為該主題建立模型。使用本課中描述的一些技術和模型構建策略。一個例子是建立一個包含家具、燈光等的客廳本體。客廳與廚房有什麼不同？與浴室呢？你如何判斷這是一個客廳而不是餐廳？使用 [Protégé](https://protege.stanford.edu/) 來構建你的本體。
|
||||
|
||||
**免責聲明**：
|
||||
此文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。我們致力於提供準確的翻譯，但請注意，自動翻譯可能包含錯誤或不準確之處。應以原始語言的文件作為權威來源。對於關鍵資訊，建議尋求專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解讀概不負責。
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,95 @@
|
|||
# 神經網絡入門：感知器
|
||||
|
||||
## [課前測驗](https://ff-quizzes.netlify.app/en/ai/quiz/5)
|
||||
|
||||
1957年，康奈爾航空實驗室的Frank Rosenblatt首次嘗試實現類似現代神經網絡的模型。他設計了一個名為「Mark-1」的硬件系統，用於識別基本幾何圖形，例如三角形、正方形和圓形。
|
||||
|
||||
| | |
|
||||
|--------------|-----------|
|
||||
|<img src='../../../../../translated_images/zh-MO/Rosenblatt-wikipedia.294821b285ac796d.webp' alt='Frank Rosenblatt'/> | <img src='../../../../../translated_images/zh-MO/Mark_I_perceptron_wikipedia.1f84eaa2d4b76ec9.webp' alt='The Mark 1 Perceptron' />|
|
||||
|
||||
> 圖片來源：[維基百科](https://en.wikipedia.org/wiki/Perceptron)
|
||||
|
||||
輸入圖像由20x20的光電陣列表示，因此神經網絡有400個輸入和一個二進制輸出。簡單的網絡包含一個神經元，也稱為**閾值邏輯單元**。神經網絡的權重類似於電位器，在訓練階段需要手動調整。
|
||||
|
||||
> ✅ 電位器是一種可以調節電路電阻的設備。
|
||||
|
||||
> 《紐約時報》當時對感知器的描述是：*一種電子計算機的胚胎，[海軍]期望它能夠行走、說話、看見、書寫、自我複製並意識到自己的存在。*
|
||||
|
||||
## 感知器模型
|
||||
|
||||
假設我們的模型有N個特徵，那麼輸入向量將是一個大小為N的向量。感知器是一種**二元分類**模型，即它可以區分兩類輸入數據。我們假設對於每個輸入向量x，感知器的輸出將是+1或-1，取決於其所屬的類別。輸出通過以下公式計算：
|
||||
|
||||
y(x) = f(w<sup>T</sup>x)
|
||||
|
||||
其中f是一個階梯激活函數
|
||||
|
||||
<!-- img src="http://www.sciweavers.org/tex2img.php?eq=f%28x%29%20%3D%20%5Cbegin%7Bcases%7D%0A%20%20%20%20%20%20%20%20%20%2B1%20%26%20x%20%5Cgeq%200%20%5C%5C%0A%20%20%20%20%20%20%20%20%20-1%20%26%20x%20%3C%200%0A%20%20%20%20%20%20%20%5Cend%7Bcases%7D%20%5C%5C%0A&bc=White&fc=Black&im=jpg&fs=12&ff=arev&edit=0" align="center" border="0" alt="f(x) = \begin{cases} +1 & x \geq 0 \\ -1 & x < 0 \end{cases} \\" width="154" height="50" / -->
|
||||
<img src="../../../../../translated_images/zh-MO/activation-func.b4924007c7ce7764.webp"/>
|
||||
|
||||
## 訓練感知器
|
||||
|
||||
為了訓練感知器，我們需要找到一個權重向量w，使得大多數值能夠正確分類，即使**誤差**最小化。這個誤差E由**感知器準則**定義如下：
|
||||
|
||||
E(w) = -∑w<sup>T</sup>x<sub>i</sub>t<sub>i</sub>
|
||||
|
||||
其中：
|
||||
|
||||
* 求和僅針對那些分類錯誤的訓練數據點i
|
||||
* x<sub>i</sub>是輸入數據，t<sub>i</sub>對應於負例和正例分別為-1或+1。
|
||||
|
||||
這個準則被視為權重w的函數，我們需要對其進行最小化。通常使用一種稱為**梯度下降**的方法，我們從某個初始權重w<sup>(0)</sup>開始，然後在每一步根據以下公式更新權重：
|
||||
|
||||
w<sup>(t+1)</sup> = w<sup>(t)</sup> - η∇E(w)
|
||||
|
||||
其中η是所謂的**學習率**，∇E(w)表示E的**梯度**。計算梯度後，我們得到：
|
||||
|
||||
w<sup>(t+1)</sup> = w<sup>(t)</sup> + ∑ηx<sub>i</sub>t<sub>i</sub>
|
||||
|
||||
Python中的算法如下：
|
||||
|
||||
```python
|
||||
def train(positive_examples, negative_examples, num_iterations = 100, eta = 1):
|
||||
|
||||
weights = [0,0,0] # Initialize weights (almost randomly :)
|
||||
|
||||
for i in range(num_iterations):
|
||||
pos = random.choice(positive_examples)
|
||||
neg = random.choice(negative_examples)
|
||||
|
||||
z = np.dot(pos, weights) # compute perceptron output
|
||||
if z < 0: # positive example classified as negative
|
||||
weights = weights + eta*weights.shape
|
||||
|
||||
z = np.dot(neg, weights)
|
||||
if z >= 0: # negative example classified as positive
|
||||
weights = weights - eta*weights.shape
|
||||
|
||||
return weights
|
||||
```
|
||||
|
||||
## 結論
|
||||
|
||||
在本課中，你學習了感知器，它是一種二元分類模型，以及如何通過使用權重向量來訓練它。
|
||||
|
||||
## 🚀 挑戰
|
||||
|
||||
如果你想嘗試自己構建感知器，可以試試[Microsoft Learn上的這個實驗](https://docs.microsoft.com/en-us/azure/machine-learning/component-reference/two-class-averaged-perceptron?WT.mc_id=academic-77998-cacaste)，它使用了[Azure ML設計器](https://docs.microsoft.com/en-us/azure/machine-learning/concept-designer?WT.mc_id=academic-77998-cacaste)。
|
||||
|
||||
## [課後測驗](https://ff-quizzes.netlify.app/en/ai/quiz/6)
|
||||
|
||||
## 回顧與自學
|
||||
|
||||
要了解如何使用感知器解決玩具問題以及實際問題，並繼續學習，請查看[感知器](Perceptron.ipynb)筆記本。
|
||||
|
||||
這裡還有一篇有趣的[關於感知器的文章](https://towardsdatascience.com/what-is-a-perceptron-basics-of-neural-networks-c4cfea20c590)。
|
||||
|
||||
## [作業](lab/README.md)
|
||||
|
||||
在本課中，我們實現了一個用於二元分類任務的感知器，並使用它來區分兩個手寫數字。在本實驗中，你需要完全解決數字分類問題，即確定給定圖像最可能對應的數字。
|
||||
|
||||
* [指導說明](lab/README.md)
|
||||
* [筆記本](lab/PerceptronMultiClass.ipynb)
|
||||
|
||||
---
|
||||
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,24 @@
|
|||
# 使用感知器進行多類別分類
|
||||
|
||||
來自 [AI for Beginners Curriculum](https://github.com/microsoft/ai-for-beginners) 的實驗課程。
|
||||
|
||||
## 任務
|
||||
|
||||
使用我們在本課程中開發的 MNIST 手寫數字二元分類程式碼，建立一個多類別分類器，能夠辨識任意數字。計算訓練集和測試集的分類準確率，並列印混淆矩陣。
|
||||
|
||||
## 提示
|
||||
|
||||
1. 對於每個數字，建立一個二元分類器的資料集，將「該數字」與「所有其他數字」進行分類。
|
||||
1. 訓練 10 個不同的感知器進行二元分類（每個數字一個感知器）。
|
||||
1. 定義一個函數，用於分類輸入的數字。
|
||||
|
||||
> **提示**：如果我們將所有 10 個感知器的權重合併成一個矩陣，我們應該能夠透過一次矩陣乘法將所有 10 個感知器應用到輸入數字上。最可能的數字可以透過對輸出進行 `argmax` 操作來找到。
|
||||
|
||||
## 起始筆記本
|
||||
|
||||
開啟 [PerceptronMultiClass.ipynb](PerceptronMultiClass.ipynb) 開始實驗。
|
||||
|
||||
---
|
||||
|
||||
**免責聲明**：
|
||||
本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。我們致力於提供準確的翻譯，但請注意，自動翻譯可能包含錯誤或不準確之處。應以原始語言的文件作為權威來源。對於關鍵資訊，建議尋求專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解讀概不負責。
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,89 @@
|
|||
# 神經網路簡介：多層感知器
|
||||
|
||||
在上一節中，你學習了最簡單的神經網路模型——單層感知器，一種線性二分類模型。
|
||||
|
||||
在本節中，我們將擴展這個模型，形成一個更靈活的框架，使我們能夠：
|
||||
|
||||
* 執行**多分類**，而不僅僅是二分類
|
||||
* 解決**回歸問題**，而不僅僅是分類問題
|
||||
* 區分非線性可分的類別
|
||||
|
||||
我們還將在 Python 中開發自己的模組化框架，讓我們能夠構建不同的神經網路架構。
|
||||
|
||||
## [課前測驗](https://ff-quizzes.netlify.app/en/ai/quiz/7)
|
||||
|
||||
## 機器學習的形式化
|
||||
|
||||
讓我們從形式化機器學習問題開始。假設我們有一個訓練數據集 **X** 和標籤 **Y**，我們需要構建一個模型 *f*，以便做出最準確的預測。預測的質量由**損失函數** ℒ 來衡量。以下是常用的損失函數：
|
||||
|
||||
* 對於回歸問題，當我們需要預測一個數值時，可以使用**絕對誤差** ∑<sub>i</sub>|f(x<sup>(i)</sup>)-y<sup>(i)</sup>|，或**平方誤差** ∑<sub>i</sub>(f(x<sup>(i)</sup>)-y<sup>(i)</sup>)<sup>2</sup>
|
||||
* 對於分類問題，我們使用**0-1損失**（本質上與模型的**準確率**相同），或**邏輯損失**。
|
||||
|
||||
對於單層感知器，函數 *f* 被定義為線性函數 *f(x)=wx+b*（其中 *w* 是權重矩陣，*x* 是輸入特徵向量，*b* 是偏置向量）。對於不同的神經網路架構，這個函數可以採用更複雜的形式。
|
||||
|
||||
> 在分類問題中，通常希望網路輸出對應類別的概率。為了將任意數字轉換為概率（例如，對輸出進行正規化），我們通常使用**softmax**函數 σ，此時函數 *f* 變為 *f(x)=σ(wx+b)*
|
||||
|
||||
在上述 *f* 的定義中，*w* 和 *b* 被稱為**參數** θ=⟨*w,b*⟩。給定數據集 ⟨**X**,**Y**⟩，我們可以計算整個數據集上的總誤差，作為參數 θ 的函數。
|
||||
|
||||
> ✅ **神經網路訓練的目標是通過調整參數 θ 來最小化誤差**
|
||||
|
||||
## 梯度下降優化
|
||||
|
||||
有一種著名的函數優化方法叫做**梯度下降**。其核心思想是，我們可以計算損失函數對參數的導數（在多維情況下稱為**梯度**），並調整參數，使誤差減少。其形式化過程如下：
|
||||
|
||||
* 用隨機值初始化參數 w<sup>(0)</sup>, b<sup>(0)</sup>
|
||||
* 重複以下步驟多次：
|
||||
- w<sup>(i+1)</sup> = w<sup>(i)</sup>-η∂ℒ/∂w
|
||||
- b<sup>(i+1)</sup> = b<sup>(i)</sup>-η∂ℒ/∂b
|
||||
|
||||
在訓練過程中，優化步驟應基於整個數據集計算（記住損失是通過所有訓練樣本的總和計算的）。然而，在實際操作中，我們會取數據集的小部分，稱為**小批量**，並基於數據子集計算梯度。由於每次隨機選取子集，這種方法被稱為**隨機梯度下降**（SGD）。
|
||||
|
||||
## 多層感知器與反向傳播
|
||||
|
||||
如上所述，單層網路能夠分類線性可分的類別。為了構建更豐富的模型，我們可以結合多層網路。數學上，這意味著函數 *f* 具有更複雜的形式，並通過多個步驟計算：
|
||||
* z<sub>1</sub>=w<sub>1</sub>x+b<sub>1</sub>
|
||||
* z<sub>2</sub>=w<sub>2</sub>α(z<sub>1</sub>)+b<sub>2</sub>
|
||||
* f = σ(z<sub>2</sub>)
|
||||
|
||||
其中，α 是**非線性激活函數**，σ 是 softmax 函數，參數 θ=<*w<sub>1</sub>,b<sub>1</sub>,w<sub>2</sub>,b<sub>2</sub>*>。
|
||||
|
||||
梯度下降算法保持不變，但計算梯度會更加困難。根據鏈式微分法則，我們可以計算導數如下：
|
||||
|
||||
* ∂ℒ/∂w<sub>2</sub> = (∂ℒ/∂σ)(∂σ/∂z<sub>2</sub>)(∂z<sub>2</sub>/∂w<sub>2</sub>)
|
||||
* ∂ℒ/∂w<sub>1</sub> = (∂ℒ/∂σ)(∂σ/∂z<sub>2</sub>)(∂z<sub>2</sub>/∂α)(∂α/∂z<sub>1</sub>)(∂z<sub>1</sub>/∂w<sub>1</sub>)
|
||||
|
||||
> ✅ 鏈式微分法則用於計算損失函數對參數的導數。
|
||||
|
||||
注意，所有這些表達式的最左部分是相同的，因此我們可以有效地從損失函數開始，沿著計算圖“向後”計算導數。因此，多層感知器的訓練方法被稱為**反向傳播**，或簡稱 'backprop'。
|
||||
|
||||
<img alt="計算圖" src="../../../../../translated_images/zh-MO/ComputeGraphGrad.4626252c0de03507.webp"/>
|
||||
|
||||
> TODO: 圖片來源
|
||||
|
||||
> ✅ 我們將在筆記本範例中更詳細地介紹反向傳播。
|
||||
|
||||
## 結論
|
||||
|
||||
在本課中，我們構建了自己的神經網路庫，並使用它完成了一個簡單的二維分類任務。
|
||||
|
||||
## 🚀 挑戰
|
||||
|
||||
在配套的筆記本中，你將實現自己的框架，用於構建和訓練多層感知器。你將能夠詳細了解現代神經網路的運作方式。
|
||||
|
||||
前往 [OwnFramework](OwnFramework.ipynb) 筆記本並完成相關內容。
|
||||
|
||||
## [課後測驗](https://ff-quizzes.netlify.app/en/ai/quiz/8)
|
||||
|
||||
## 回顧與自學
|
||||
|
||||
反向傳播是 AI 和 ML 中常用的算法，值得[深入研究](https://wikipedia.org/wiki/Backpropagation)。
|
||||
|
||||
## [作業](lab/README.md)
|
||||
|
||||
在本次實驗中，你需要使用本課中構建的框架來解決 MNIST 手寫數字分類問題。
|
||||
|
||||
* [說明](lab/README.md)
|
||||
* [筆記本](lab/MyFW_MNIST.ipynb)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,183 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 使用我們自己的框架進行 MNIST 數字分類\n",
|
||||
"\n",
|
||||
"來自 [AI 初學者課程](https://github.com/microsoft/ai-for-beginners) 的實驗作業。\n",
|
||||
"\n",
|
||||
"### 讀取數據集\n",
|
||||
"\n",
|
||||
"此程式碼從網路上的資料庫下載數據集。你也可以手動從 AI 課程資料庫的 `/data` 目錄中複製數據集。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" % Total % Received % Xferd Average Speed Time Time Time Current\n",
|
||||
" Dload Upload Total Spent Left Speed\n",
|
||||
"\n",
|
||||
" 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\n",
|
||||
"100 9.9M 100 9.9M 0 0 9.9M 0 0:00:01 --:--:-- 0:00:01 15.8M\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!rm *.pkl\n",
|
||||
"!wget https://raw.githubusercontent.com/microsoft/AI-For-Beginners/main/data/mnist.pkl.gz\n",
|
||||
"!gzip -d mnist.pkl.gz"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pickle\n",
|
||||
"with open('mnist.pkl','rb') as f:\n",
|
||||
" MNIST = pickle.load(f)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"labels = MNIST['Train']['Labels']\n",
|
||||
"data = MNIST['Train']['Features']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"讓我們看看我們擁有的數據形狀：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(42000, 784)"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"data.shape"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 分割資料\n",
|
||||
"\n",
|
||||
"我們將使用 Scikit Learn 將資料分割為訓練集和測試集：\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Train samples: 33600, test samples: 8400\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"\n",
|
||||
"features_train, features_test, labels_train, labels_test = train_test_split(data,labels,test_size=0.2)\n",
|
||||
"\n",
|
||||
"print(f\"Train samples: {len(features_train)}, test samples: {len(features_test)}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 指示\n",
|
||||
"\n",
|
||||
"1. 從課程中取得框架代碼，並將其貼到此筆記本中，或（更好）貼到一個獨立的 Python 模組中\n",
|
||||
"1. 定義並訓練單層感知器，觀察訓練和驗證的準確率\n",
|
||||
"1. 嘗試理解是否發生了過擬合，並調整層的參數以提高準確率\n",
|
||||
"1. 重複上述步驟，訓練兩層和三層感知器。嘗試在層之間使用不同的激活函數進行實驗\n",
|
||||
"1. 嘗試回答以下問題：\n",
|
||||
" - 層間的激活函數是否影響網絡性能？\n",
|
||||
" - 這個任務是否需要兩層或三層的網絡？\n",
|
||||
" - 在訓練網絡時是否遇到任何問題？特別是當層數增加時。\n",
|
||||
" - 網絡的權重在訓練過程中如何變化？你可以繪製權重最大絕對值與訓練輪次的關係圖來理解它們的關聯性。\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n---\n\n**免責聲明**： \n本文件已使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。儘管我們努力確保翻譯的準確性，但請注意，自動翻譯可能包含錯誤或不準確之處。原始文件的母語版本應被視為權威來源。對於關鍵信息，建議使用專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解釋不承擔責任。\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.7.4 64-bit (conda)",
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5"
|
||||
}
|
||||
},
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.5"
|
||||
},
|
||||
"orig_nbformat": 2,
|
||||
"coopTranslator": {
|
||||
"original_hash": "6fa055f484eb5d6bdf41166a356d3abf",
|
||||
"translation_date": "2025-08-28T11:37:26+00:00",
|
||||
"source_file": "lessons/3-NeuralNetworks/04-OwnFramework/lab/MyFW_MNIST.ipynb",
|
||||
"language_code": "mo"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
# 使用我們自己的框架進行 MNIST 分類
|
||||
|
||||
來自 [AI for Beginners Curriculum](https://github.com/microsoft/ai-for-beginners) 的實驗作業。
|
||||
|
||||
## 任務
|
||||
|
||||
使用 1 層、2 層和 3 層的感知器解決 MNIST 手寫數字分類問題。使用我們在課程中開發的神經網路框架。
|
||||
|
||||
## 啟動筆記本
|
||||
|
||||
通過打開 [MyFW_MNIST.ipynb](../../../../../../lessons/3-NeuralNetworks/04-OwnFramework/lab/MyFW_MNIST.ipynb) 開始實驗。
|
||||
|
||||
## 問題
|
||||
|
||||
完成此實驗後，嘗試回答以下問題：
|
||||
|
||||
- 層間的激活函數是否會影響網路性能？
|
||||
- 這個任務是否需要 2 層或 3 層的網路？
|
||||
- 在訓練網路時是否遇到任何問題？尤其是當層數增加時。
|
||||
- 網路的權重在訓練過程中如何變化？您可以繪製權重的最大絕對值與訓練輪次的關係圖來理解它們的關聯性。
|
||||
|
||||
**免責聲明**：
|
||||
本文件使用 AI 翻譯服務 [Co-op Translator](https://github.com/Azure/co-op-translator) 進行翻譯。我們致力於提供準確的翻譯，但請注意，自動翻譯可能包含錯誤或不準確之處。應以原始語言的文件作為權威來源。對於關鍵資訊，建議尋求專業人工翻譯。我們對因使用此翻譯而引起的任何誤解或錯誤解讀概不負責。
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Loading…
Reference in New Issue