RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

54 Commits

Author SHA1 Message Date
Adrián Chaves 1d11ea3a54
Update practices.rst 2024-04-11 12:19:32 +02:00
Adrián Chaves 70ba3a0868
SPM → Zyte API (#6163) 2023-11-30 11:01:22 +01:00
Andrey Rakhmatullin 76eba9977b
Skip or fix failing code block tests (#5826) 2023-02-14 09:40:38 +01:00
pankaj1707k c1bbb299d7
Add and run pre-commit hook 'blacken-docs' 
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
 2023-02-01 16:30:57 +05:30
Yash 319e67f779 documentation update for multiple spiders 
i noticed passing settings to configure logging function made weird output go away. checked documentation and it says first parameter is settings file. Is this correct?
 2022-04-12 11:42:22 -05:00
Florentin ccdbb795ff Recommend Common Crawl instead of Google Cache 2022-03-01 22:01:55 +01:00
Andrey Rakhmatullin 6eaceec735 Implement docs suggestions. 2021-12-31 20:14:24 +05:00
Andrey Rakhmatullin 940cc0776f Add docs about TWISTED_REACTOR and other per-process settings. 2021-12-24 17:12:50 +05:00
Andrey Rakhmatullin cd17c829cf Revert "Move documentation about avoiding bans into a topic of its own (#4039)" 
This reverts commit 2d2581c68f.
 2021-08-23 19:55:35 +05:00
Adrián Chaves 2d2581c68f
Move documentation about avoiding bans into a topic of its own (#4039) 2021-08-18 12:46:42 +02:00
Andrey Rahmatullin 5b78a64fca
Merge pull request #5070 from ric2b/patch-1 2021-04-15 14:50:59 +05:00
Ricardo Amendoeira 8603f9d7a5
Apply changes to other examples in the same section. 2021-04-06 20:23:07 +01:00
Ricardo Amendoeira b247fa9982
Include loading settings in `Running multiple spiders in the same process` section 
The example in the documentation doesn't take into account the project settings
 2021-03-29 01:48:28 +01:00
Adrián Chaves f30f53b3cc Scrapinghub → Zyte 2021-02-02 15:03:20 +01:00
Eugenio Lacuesta 04b6295a69
Docs: replace deprecated FEED_* settings 2020-04-17 20:50:17 -03:00
Adrián Chaves e84cb18ca0 Use InterSphinx to link to the Twisted documentation 2019-11-18 15:50:45 +01:00
Deschner, Magdalena c44d49b238 minor PEP8 style changes 2019-07-17 13:13:52 +02:00
Deschner, Magdalena 6660020ebb remove detailed description about individual settings 2019-07-17 11:30:02 +02:00
Deschner, Magdalena e892a484e8 add instructions about how to define output file when running scrapy from script instead of cmd 2019-07-16 13:53:56 +02:00
Adrián Chaves 75d6f56c8a Switch from ` to `` where inline code formatting is desired 2019-03-01 16:56:58 +01:00
Aditya 9d9d83a8c3 Use https links wherever possible 2017-10-28 16:24:40 +05:30
Aditya 9dd680d5c9 Use https for external links wherever possible in docs 2017-10-26 23:33:45 +05:30
Pablo Hoffman a429d78019 update scrapinghub.com urls to use https 2017-08-24 16:03:36 -03:00
Martín Gaitán 4bc0c6b0f4 Update practices.rst 
fix a typo
 2017-04-26 14:12:57 +05:00
Pablo Hoffman d62776a858 mention scrapoxy in best practices doc 2016-11-16 12:19:32 -03:00
Kenneth Schnall 2489f84d39 Update practices.rst 2016-07-12 15:34:50 -04:00
nyov 5876b9aa30 Update documentation links 2016-03-03 16:28:33 +00:00
David Chen 0025d5a943 Fixed minor grammar issues. 2015-11-16 07:30:17 +08:00
Aivars Kalvāns b8b1e8e544 Make common practices sample code match the comments 2015-08-19 16:54:10 +03:00
Mikhail Korobov 9a787893e3 (backwards-incompatible) allow to pass settings=None to configure_logging 
* use explicit argument for disabling root handler;
* handle LOG_STDOUT even if install_root_handler is False
 2015-06-08 19:54:18 +05:00
Mikhail Korobov d047665c02 make "settings" argument optional for Crawler, CrawlerRunner and CrawlerProcess 2015-06-06 03:23:13 +05:00
Pablo Hoffman 545c4224f9 update old crawlera link 2015-05-25 16:01:54 -03:00
Julia Medina c1634e4914 Add CrawlerProcess to "Running multiple spiders[...]" doc section 2015-04-29 23:09:20 -03:00
Julia Medina d6a06a769f Add CrawlerProcess to "Run Scrapy from a script" doc section 2015-04-29 23:09:20 -03:00
Julia Medina bd0b639b21 Fix logging usage across docs 2015-04-22 17:24:41 -03:00
Mikhail Korobov 7644813286 allow Crawler, CrawlerRunner and CrawlerProcess to accept dicts instead of Setting objects 2015-04-14 23:26:05 +05:00
Mikhail Korobov 5ac91e4883 DOC remove Dynamic Creation of Item Classes section 
It was a hack, and dicts-as-items cover most use cases.

Dicts don't allow to attach metadata to fields,
but e.g. adding "_meta" key and removing it in a custom serializer
is no worse than creating classes dynamically.
 2015-03-23 18:11:35 +05:00
Mikhail Korobov 817dbc6cbd DOC mention dicts in documentation; explain better what are Items for 2015-03-19 05:16:14 +05:00
Capi Etheriel 4bc14da59e Updates documentation on dynamic item classes. 
Fixes #398
 2015-01-19 17:21:56 -02:00
Mikhail Korobov bc0f481a73 DOC bring back notes about multiple spiders per process because it is now documented how to do that 2014-09-21 07:12:01 +06:00
Julia Medina 3547ca6e61 Add example on running spiders outside projects 2014-08-14 11:50:33 -03:00
Julia Medina d40273561d CrawlerProcess cleanup changes 2014-08-12 14:02:55 -03:00
Mikhail Korobov 2d3803672b DOC use top-level shortcuts in docs 2014-04-15 01:09:35 +06:00
Cash Costello c532e13dcd Added missing word in practices.rst 2014-02-27 13:08:36 -02:00
Capi Etheriel 1ec1100b27 fixes dynamic itemclass example usage of type() 2014-02-18 20:59:05 -03:00
Pablo Hoffman 8b9526a8f6 Merge pull request #400 from irgmedeiros/patch-2 
Update the second code example
 2013-10-07 07:57:18 -07:00
Loren Davie 8af0e89e85 Corrected typo. 2013-09-29 17:06:46 -04:00
Loren Davie f49f5724d5 Added dynamic creation of item classes to practices.rst. 2013-09-28 09:00:48 -04:00
irgmedeiros 9b50409986 Update the second code example 
Update the second code example to reflect the last change in the first example.
 2013-09-27 18:22:33 -03:00
irgmedeiros d9e0fdc9aa Update practices.rst 
With this modification scrapy runs the spider with project settings. The previous example ran only with default settings resulting in ignoring all user settings as pipelines for example.
 2013-09-27 17:56:30 -03:00