RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,059 Commits 49 Branches 132 Tags 179 MiB
Commit Graph

10059 Commits

Author SHA1 Message Date
Andrey Rakhmatullin cddb8c15d6
Merge pull request #5926 from GeorgeA92/periodic_log_2 
periodic_log: implemented as separate extension
 2023-08-30 22:29:54 +04:00
Andrey Rakhmatullin 0e7fc60b89
Merge pull request #6021 from dveselov/master-1 
Add .webp to link extractor IGNORED_EXTENSIONS list
 2023-08-21 22:04:03 +04:00
Georgiy Zatserklianyi fe02642980 Merge remote-tracking branch 'origin/periodic_log_2' into periodic_log_2 2023-08-21 17:15:42 +03:00
Georgiy Zatserklianyi 7e542846e4 periodic_log: `datetime.utcnow()` changed to `datetime.now(tz=timezone.utc)` 2023-08-21 17:13:18 +03:00
Georgiy Zatserklianyi 1f03cb1419 Merge remote-tracking branch 'upstream/master' into periodic_log_2 2023-08-21 17:07:18 +03:00
Laerte Pereira df2163ce6a
Remove datetime.utcnow() usage (#6014) 2023-08-21 15:51:49 +02:00
Dima Veselov 7355741c7d
Add .webp to IGNORED_EXTENSIONS 2023-08-20 22:38:56 +03:00
Andrey Rakhmatullin 3318971512
Merge pull request #6013 from Laerte/master 
Handle Tuple type on getdictorlist method, bump 3.12 python version
 2023-08-14 19:55:38 +04:00
Laerte Pereira b06936f111 Handle Tuple type on getdictorlist method, bump 3.12 python version 2023-08-14 10:33:48 -03:00
Georgiy Zatserklianyi ac1694a9ad
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:49:09 +03:00
Georgiy Zatserklianyi d67be20b2d
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:48:58 +03:00
Georgiy Zatserklianyi 3a4a949f9d
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:48:48 +03:00
Georgiy Zatserklianyi e6bd9829bd
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:48:31 +03:00
Georgiy Zatserklianyi 2f094a7a5c
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:48:14 +03:00
Andrey Rakhmatullin 7c497688f8
Merge pull request #6005 from wRAR/typing-crawler 
More typing for scrapy/*.py
 2023-08-11 14:41:47 +04:00
Adrián Chaves 8a0108b19c
Merge branch 'master' into typing-crawler 2023-08-11 10:42:45 +02:00
Andrey Rakhmatullin 34d050cfe5
Remove deprecated CrawlerRunner.spiders. (#6010) 2023-08-11 10:41:05 +02:00
Andrey Rakhmatullin 44b15c3004 Remove typing for CrawlerRunner.spider_loader. 2023-08-10 23:17:06 +04:00
Andrey Rakhmatullin 9df67a554e Add RequestFingerprinterProtocol. 2023-08-10 22:53:22 +04:00
Georgiy Zatserklianyi 736a4b615c
Update scrapy/settings/default_settings.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-10 21:34:35 +03:00
Georgiy Zatserklianyi f05657e542 periodic_log: interval check moved 2023-08-10 21:31:10 +03:00
Laerte Pereira 9e74748fca
Remove extra spider parameter in item pipeline docs (#6009) 2023-08-10 13:48:43 +02:00
Laerte Pereira 4a090d951a
Remove deprecated PythonItemExporter.binary (#6007) 2023-08-10 07:36:42 +02:00
Andrey Rakhmatullin b50268c100 Full typing for scrapy/link.py. 2023-08-09 23:56:16 +04:00
Andrey Rakhmatullin e6d919497d More typing for scrapy/core/scheduler.py. 2023-08-09 23:56:16 +04:00
Andrey Rakhmatullin 7dca18e2e7 Full typing for scrapy/responsetypes.py. 2023-08-09 23:56:16 +04:00
Andrey Rakhmatullin c5885fc13b Full typing for scrapy/exceptions.py. 2023-08-09 23:56:16 +04:00
Andrey Rakhmatullin 9960c62b87 Full typing for scrapy/logformatter.py. 2023-08-09 23:56:16 +04:00
Andrey Rakhmatullin 89503ae3f1 Full typing for scrapy/dupefilters.py. 2023-08-09 23:56:16 +04:00
Andrey Rakhmatullin dc6e142096 Full typing for scrapy/spiderloader.py. 2023-08-09 23:56:16 +04:00
Andrey Rakhmatullin 084a9ba076 Full typing for scrapy/crawler.py and scrapy/spiders/__init__.py. 2023-08-09 23:56:16 +04:00
Andrey Rakhmatullin 85696d7bab
Merge pull request #6003 from wRAR/typing-utils-2 
Typing for scrapy/utils, second pass
 2023-08-09 23:56:03 +04:00
Andrey Rakhmatullin 8050257c14 Small cleanup. 2023-08-09 23:17:32 +04:00
Andrey Rakhmatullin 53539483c3 Refactor _StreamReader.read(). 2023-08-07 15:13:20 +04:00
Andrey Rakhmatullin 2c7484d433 Merge remote-tracking branch 'origin/master' into typing-utils-2 2023-08-07 14:55:38 +04:00
Andrey Rakhmatullin 110d5fffb4
Update tool versions. (#6002) 2023-08-07 10:57:48 +02:00
Andrey Rakhmatullin 23af21491d Move definitions around to woark around a pypy3.8 bug. 2023-08-07 00:21:06 +04:00
Andrey Rakhmatullin 644a71bfd4 Use ftp:// URLs in FTP tests. 2023-08-07 00:17:52 +04:00
Andrey Rakhmatullin 471281d29e Fixes for scrapy/utils/iterators.py typing. 2023-08-06 23:05:02 +04:00
Andrey Rakhmatullin f5f593e5f5 Remove a workaround for a w3lib typing bug. 2023-08-06 17:46:28 +04:00
Andrey Rakhmatullin e2adec629b Fix regressions in typing. 2023-08-06 17:44:24 +04:00
Andrey Rakhmatullin 518e56046e Check for async callbacks in contracts. 2023-08-06 17:28:34 +04:00
Andrey Rakhmatullin 66bad1150c Add more typing for scrapy/utils/signal.py. 2023-08-06 00:35:33 +04:00
Andrey Rakhmatullin 9fe662d856 Add typing for scrapy/utils/testproc.py. 2023-08-06 00:35:33 +04:00
Andrey Rakhmatullin d1f87e4f08 More typing for scrapy/utils/iterators.py. 2023-08-06 00:35:33 +04:00
Andrey Rakhmatullin c43798cb9b More typing for scrapy/utils/defer.py and scrapy/utils/spider.py. 2023-08-06 00:35:33 +04:00
Andrey Rakhmatullin d015329d75 Add more typing for scrapy/utils/log.py. 2023-08-06 00:35:33 +04:00
Andrey Rakhmatullin d31829b72f More typing for scrapy/utils. 2023-08-06 00:35:33 +04:00
Andrey Rakhmatullin 88327c7c58 Bump version: 2.9.0 → 2.10.0 2023-08-04 17:23:30 +04:00
Andrey Rakhmatullin 4d288c8bf3
Merge pull request #6001 from scrapy/relnotes-2.10 
Add release notes for 2.10.0.
 2023-08-04 17:22:16 +04:00