RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,612 Commits 49 Branches 132 Tags 137 MiB
Commit Graph

1702 Commits

Author SHA1 Message Date
Adrián Chaves 068af85722
Add release notes for Scrapy 2.8 (#5799) 2023-02-02 04:54:02 +01:00
Adrián Chaves 9272c4af0c Merge remote-tracking branch 'scrapy/master' into no-callback 2023-01-25 17:41:41 +01:00
Laerte Pereira b611848029
fix(docs): Change `versionchanged` value 2023-01-20 08:35:17 -03:00
Laerte Pereira 973f0cf567
fix: line break 2023-01-20 08:23:05 -03:00
Laerte Pereira 8270df754d
Set `FEED_EXPORT_ENCODING='utf-8'` in the default template 2023-01-20 07:55:16 -03:00
Adrián Chaves a493464942 Update the screenshot pipeline code example 2023-01-19 19:53:53 +01:00
Adrián Chaves 50500a6b28 Implement a NO_CALLBACK value for Request.callback 2023-01-19 17:14:18 +01:00
Andrey Rakhmatullin f87d061069
Merge branch 'master' into download-delay-documentation 2023-01-13 00:22:48 +05:00
Josh Soref b62aacfee3 spelling: initializing 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 21:33:21 -05:00
Yohanna Lisnichuk b63ca6f834 docs: use the right notes directive 2022-12-15 13:20:45 -03:00
Yohanna Lisnichuk 96e526aad8
docs: change DOWNLOAD_DELAY description format 
To be consistent with the other settings' descriptions
 2022-12-15 10:01:55 -03:00
Andrey Rahmatullin d19a216e10
Merge branch 'master' into pathlib 2022-11-25 18:20:25 +05:00
Pablo Hoffman c04ccbceb9 doc: add instructions to debug Scrapy spiders in Visual Studio Code 2022-11-21 15:49:33 +01:00
Hassan Shoayb 29bf7f5a6c
broad-crawls.rst: fix a typo (#5714) 2022-11-14 09:45:00 +01:00
Magnus Offermanns fd692f3091
Prevent running the -O and -t command-line options together (#5605) 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2022-10-27 14:43:31 +02:00
Mikhail Korobov 82f25bc44a
Merge pull request #5681 from mattkohl-flex/master 
Documentation: typo fixes
 2022-10-18 15:27:52 +05:00
Matthew Donoughe 40d9ca3bdd
use pathlib 2022-10-17 17:40:10 -04:00
Adrián Chaves 06c8f673af
2.7 release notes (#5680) 
* Fix the display name of documented fingerprinter class methods

* Initial draft for the Scrapy 2.7 release notes

* Update VERSION and PREVIOUS_VERSION references

* Clarify the restrictions lifted for item field output names

* Fix the description of the BOM bug fix

* Fix the note about changes in MIME sniffing

* Fix typo

* Extend highlights

* Fyx typo
 2022-10-17 18:04:29 +05:00
mattkohl-flex c49764ffd7 typo fixes 2022-10-17 11:15:17 +01:00
Andrey Rakhmatullin 62cc26e209 Change TWISTED_REACTOR in the default template. 2022-10-14 22:03:54 +06:00
Mikhail Korobov 4f296b61b0
Merge pull request #5608 from zaid-ismail031/master 
Changed incorrect info in documentation regarding the return type of parse method.
 2022-10-03 00:19:08 +05:00
Gábor Lipták 79a4bc3da0
Cleanup METHOD_SSLv3 
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
 2022-09-25 17:01:43 -04:00
Tim B 5f19420211
Documented how settings must be picklable (#5629) 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Laerte Pereira <5853172+Laerte@users.noreply.github.com>
 2022-09-21 08:27:27 +02:00
zaid-ismail031 c49b5aaf77 Changed incorrect information regarding the return type of parse/request callback method. 2022-08-26 21:17:31 +02:00
Andrey Rahmatullin 83ecdf1bca
Update docs/topics/components.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2022-07-27 23:12:31 +05:00
Mikhail Korobov 4dde2f2c36 Merge branch 'master' into asyncio-parse-fixes 2022-07-27 09:32:54 +05:00
Adrián Chaves 00ccb02f1f Merge remote-tracking branch 'upstream/2.6' 2022-07-25 14:13:22 +02:00
Andrey Rahmatullin a0681fb811
Merge branch 'master' into asyncio-parse-asyncgen-proper-rebased 2022-07-22 18:30:38 +05:00
Mikhail Korobov f60c7ae768
Fixed heading levels in downloader middleware docs (#5567) 2022-07-20 11:01:22 +02:00
Mikhail Korobov 77a3b02523
Merge pull request #5553 from Rotzbua/fix-doc-typo 
Fix doc: `scrapy.exporter` to `scrapy.exporters`
 2022-07-17 00:21:50 +05:00
Rotzbua 1c7ed4f2e5
[doc] Remove incompatible web service project 
* Abandoned since 2017
* Not compatible with Python3
 2022-07-12 13:09:20 +02:00
Rotzbua 09c3a4ad08
Fix doc: `scrapy.exporter` to `scrapy.exporters` 2022-07-12 12:45:15 +02:00
Mikhail Korobov 500dae82e2
Merge pull request #3696 from Gallaecio/allow-customizing-export-column-names 
Allow customizing export column names
 2022-06-27 00:26:05 +05:00
Adrián Chaves 34e4ed72ea Document how DOWNLOAD_DELAY affects per-domain concurrency 2022-06-21 12:46:54 +02:00
Adrián Chaves 4ef71829b2
If TWISTED_REACTOR is None, reuse any pre-installed reactor (#5528) 2022-06-17 10:37:27 +02:00
Michel Ace 6e878490e8
Support and prefer the .jsonl file extension (#4848) 2022-06-17 08:37:14 +02:00
Adrián Chaves 3e994bda45 Update for Python 3.7+ 2022-06-16 20:53:14 +02:00
Adrián Chaves e8503217fb Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2022-06-16 20:19:52 +02:00
Laerte Pereira 407562b38b
Drop Python 3.6 support (#5514) 
* chore: Drop Python 3.6 support

* Attend PR comments

* Tweak versions

* Update dependencies version

* fix: Ubuntu workflow

* fix windows workflow

* chore: Remove comment

* update `install_requires` dependencies versions

* move lxml to main pinned requirements

* Attend code-review comments

* remove non-pinned 3.7 from windows workflow

* simplify condition

* lint

* remove paragraph

* refactor

* remove leftover
 2022-06-09 00:25:03 -03:00
Adrián Chaves c5627af15b
Centralize request fingerprints (#4524) 
Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
 2022-06-07 18:44:54 +02:00
Mikhail Korobov 517cbc8d71
Merge pull request #5508 from mtabbasi/thumb_path-access-item 
pass on item to thumb_path function as additional argument
 2022-06-05 03:42:45 +05:00
Mikhail Korobov bafe874ecc
Merge pull request #5358 from burakcank/signal_coroutine 
Document coroutines for signals.
 2022-05-28 13:38:05 +05:00
Mohammadtaher Abbasi f39def4492 add docs 2022-05-25 23:57:38 +04:30
Yash 319e67f779 documentation update for multiple spiders 
i noticed passing settings to configure logging function made weird output go away. checked documentation and it says first parameter is settings file. Is this correct?
 2022-04-12 11:42:22 -05:00
Andrey Rahmatullin 8c3e5a2421
Merge pull request #5457 from d00399830/typo_fixes_sm 
Fix capitalization of JavaScript within the Developer Tools documentation
 2022-03-23 11:06:54 +05:00
Víctor Ruiz 2227be7af6
Fix a typo in the HTTP cache documentation (#5455) 2022-03-22 15:21:16 +01:00
D00399830 fcf3d8e0a0 Updated the documentation for developer tools to have JavaScript instead of Javascript, as JavaScript is the more correct way to write it 2022-03-21 14:09:31 -06:00
Adrián Chaves b95c634b86 Document how to enforce Scrapy versions on Scrapy components 2022-03-17 22:23:25 +01:00
Eugene 9a28eb0bad
Suggest installing the brotli package instead of brotlipy (#4267) 2022-03-17 05:39:54 +01:00
Adrián Chaves b78e6915c6 Clarify that without async-to-sync conversions items yielded before an exception are processed 2022-03-16 20:17:25 +01:00