RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,620 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

2755 Commits

Author SHA1 Message Date
Andrey Rakhmatullin ce56d227e5 Fix a typo that broke PDF builds. 2024-11-18 15:39:49 +05:00
Adrián Chaves 10089c6fe2
2.12 release notes (#6226) 
* Cover 2.12 in the release notes up to 9bb973dc54

* Add one more highlight

* Better merge of the news entries.

* Cover 2.12 in the release notes up to 642af40.

* Cover 2.12 in the release notes up to 7a0a34b.

* Cover 2.12 in the release notes up to b4bad97.

* Add not yet merged PRs #6463, #6507, #6511 to the 2.12 release notes.

* Cover 2.12 in the release notes up to d85c39f, small fixes.

* Cover 2.12 in the release notes up to d215669.

* Cover #6527 in the release notes.

* Address PR feedback.

* Cover recent PRs.

* Finalize the 2.12.0 release notes, small additional fixes.

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-11-18 13:07:32 +05:00
Andrey Rakhmatullin ab5cb7c7d9
Merge pull request #6540 from wRAR/build_from_settings 
Remove build_from_settings() and deprecate from_settings() code
 2024-11-14 20:40:54 +05:00
Robert Palmer 261c4b61dc
Enhancement: Update docs to include IgnoreRequest details (#6506) 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2024-11-13 22:47:39 +05:00
Andrey Rakhmatullin 499e7e8aa6 Add from_crawler() to components that only had from_settings(). 2024-11-12 21:12:32 +05:00
Adrián Chaves dc3ebb6cf7 Refactor the docs 2024-11-07 16:38:48 +01:00
Adrián Chaves 2a4b7fe0f8 dont_throttle → autothrottle_dont_adjust_delay 2024-11-07 16:17:16 +01:00
Adrián Chaves b244ea7ac0 Add the missing import to the docs example 2024-11-05 20:05:58 +01:00
Adrián Chaves 5862216bb1 Fix docs example 2024-11-05 19:55:28 +01:00
Adrián Chaves f57fc454be Replace Slot.throttle with Request.meta['dont_throttle'] 2024-11-05 19:44:30 +01:00
Andrey Rakhmatullin e7f5ae0b34
Update the outdated Item docstring. (#6427) 
* Update the outdated Item doscstring.

* Fix the reference links in items.html.
 2024-11-04 16:17:56 +05:00
Andrey Rakhmatullin ce5a132f12
Run and fix linkcheck. (#6524) 2024-11-04 11:40:07 +01:00
Rohit Kumar Singh 7701e590fb
Documentation added for Spider State in extensions.rst (#6522) 
* Documentation added for Spider State in extensions.rst

* Made correction in documentation for Spiderstate

* Added appropriate intro for Spider state extension

* Added reference for spiderstate extension

* Added Spiderstate extension hyperlink refrence in jobs.rst
 2024-11-02 10:45:27 +05:00
Andrey Rakhmatullin d85c39f5bc
Deprecation removals. (#6500) 
* Deprecation removals.

* Clean up the default pytest filterwarnings.

* Remove test_get_images_old().

* Redo boto-requiring test filtering.

* Remove an unused function.

* Improve the Crawler.crawl() error message.

* Fix the test.
 2024-10-31 18:06:22 +05:00
Andrey Rakhmatullin 0523e1616d
Explictly set html_baseurl on RTD. (#6507) 2024-10-29 10:16:03 +01:00
Andrey Rakhmatullin 677e977207 Remove dead links to the Reppy doc from the release notes. 2024-10-18 00:03:32 +05:00
Andrey Rakhmatullin 5759b3f0f2 Drop Reppy. 2024-10-17 23:41:23 +05:00
Julian Ste c8e87ab21a
Fixed typos (#6497) 2024-10-17 17:03:16 +02:00
Vsevolod Breus 5391663072
Drop Python 3.8 Support (#6472) 2024-10-16 10:03:16 +02:00
mmoriniere 46cddc6ecf
Ignore SyntaxError as well when SPIDER_LOADER_WARN_ONLY is set to True (#6484) 2024-10-02 10:04:03 +02:00
Laerte Pereira 4f9b2343c0 Merge branch 'master' of github.com:Laerte/scrapy 2024-08-27 07:16:10 -03:00
Laerte Pereira 3c2a9fa262 update docs 2024-08-27 07:16:01 -03:00
Laerte Pereira f68f29dd13
Update docs/topics/contracts.rst 2024-08-26 23:37:57 -03:00
Laerte Pereira b85e5a66ed Add support for meta in Spider Contracts 2024-08-26 23:21:09 -03:00
Georgiy Zatserklianyi 6ce0342beb
Allow yielding items from start_requests (#6417) 
Co-authored-by: Georgiy Zatserklianyi <george.zatseklyany@gmail.com>
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-08-26 20:53:06 +02:00
mlmsmith af15bd1dad
minor changes to items section of docs (#6462) 2024-08-19 13:55:09 +02:00
mlmsmith b9ef1326a5
Proofread the commands documentation (#6449) 2024-08-01 09:29:11 +02:00
Adrián Chaves 03a15ced4f
Do not suggest logging dropped items twice (#6448) 
Co-authored-by: Kevin Lloyd Bernal <kevinoxy@gmail.com>
 2024-07-31 11:37:19 +02:00
mlmsmith e376c0b31a
Tutorial edits (#6440) 2024-07-24 06:40:01 +02:00
guillermo-bondonno 026d606528
clean closespider_pagecount_no_item docs section 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-07-12 08:09:03 -03:00
guillermo-bondonno 8646d2ec7b fix docs detail 2024-07-10 11:44:44 -03:00
guillermo-bondonno 59782d7308 update docs 2024-07-10 11:08:22 -03:00
mlmsmith d8ecd28c55
Documentation improvements (#6429) 2024-07-04 12:16:26 +02:00
mlmsmith 326e323e11
Apply grammar fixes (#6411) 2024-06-21 12:24:10 +02:00
Andrey Rakhmatullin 365c9e62ad
Removing empty example reference (#6402) 
Co-authored-by: Michael Duane Mooring <mikeumus@gmail.com>
 2024-06-17 11:37:11 +02:00
Laerte Pereira 144ff6c756 Document missing parts of response.json method 2024-06-05 21:09:10 -03:00
Laerte Pereira 62c89aaf05
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:45 -03:00
Laerte Pereira 8ec67ca230
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:35 -03:00
Laerte Pereira 17e623cf0c
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:24 -03:00
Laerte Pereira 3f66b66e3f fix: checks 2024-05-22 22:01:55 -03:00
Laerte Pereira dc6a495fee
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-22 18:51:02 -03:00
Laerte Pereira 8210fae25a
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-22 18:50:50 -03:00
Laerte Pereira e676cd3ce0 docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunner examples 2024-05-22 07:55:53 -03:00
Adrián Chaves 812fd2368f
Allow user-defined secure cookies (#6357) 2024-05-15 11:48:43 +02:00
Adrián Chaves d2f1e00a6a
Merge 2.11.2 changes (#6363) 2024-05-14 18:54:11 +02:00
Sanchay Kumar c9ef520936
Add Downloader.get_slot_key() without a spider parameter (#6352) 2024-05-10 14:26:45 +02:00
Adrián Chaves fe163d98ea
Fix typo 2024-05-08 20:41:20 +02:00
aisha-partha 2e13a9b8e1 Update METAREFRESH_IGNORE_TAGS default value in documentation 2024-05-08 23:33:13 +05:30
aisha-partha 3590a1f66b Closes #6342. Update documentation on METAREFRESH_IGNORE_TAGS default value change 2024-05-08 23:23:17 +05:30
aisha-partha 180bc9bad7 Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default 2024-05-08 22:06:46 +05:30