RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 180 MiB
Commit Graph

1854 Commits

Author SHA1 Message Date
Georgiy Zatserklianyi 8b6a50a935 periodic_log: docs added 2023-08-04 11:12:07 +03:00
Adrián Chaves 72de48be6d asyncio: cover accidental bad reactor installation, sort sections, reword the Windows section 2023-08-04 09:56:30 +02:00
Andrey Rakhmatullin 7c6aaedda2
Merge pull request #5846 from jxlil/docs/spiders 
Added documentation for Spider.update_settings
 2023-08-02 21:09:44 +04:00
Andrey Rakhmatullin af1be835e4
Apply suggestions from code review 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-02 19:46:16 +04:00
Andrey Rakhmatullin 72462a53e2 Add more docs for update_settings(). 2023-08-02 12:32:53 +04:00
Andrey Rakhmatullin 06ebdee35d
Merge pull request #5950 from wRAR/addons 
Add-ons
 2023-08-01 22:18:57 +04:00
Andrey Rakhmatullin cf55eb05f5 Fix a typo. 2023-08-01 13:30:56 +04:00
Andrey Rakhmatullin 41a4a163e3 Handle NotConfigured in add-ons. 2023-07-31 21:09:18 +04:00
Andrey Rakhmatullin b67a81b81d Use the MyHandler class directly. 2023-07-31 20:25:56 +04:00
Andrey Rakhmatullin d8c5b41559 Add more addon tests. 2023-07-31 19:09:21 +04:00
Andrey Rakhmatullin bddbbc522a
Merge pull request #5979 from BurnzZ/closespider-timeout-no-item 2023-07-28 12:56:44 +04:00
Andrey Rakhmatullin 3ba2dc4d68 Fixes for addon docs. 2023-07-24 17:49:04 +04:00
Kevin Lloyd Bernal 9a1bf40c2f expose AWS_REGION_NAME in S3FeedStorage 2023-07-24 16:37:52 +08:00
Mikhail Korobov af2aa4b421
Merge pull request #5847 from namelessGonbai/feat/FixFeedExport 
Fix FeedExporter not to export empty file
 2023-07-22 23:18:36 +05:00
Andrey Rakhmatullin 005c8cc5f0 Unify the "add-on" spelling. 2023-07-19 13:15:35 +04:00
Andrey Rakhmatullin 0a25a300cf Fix docs. 2023-07-18 20:29:42 +04:00
Andrey Rakhmatullin 90dae3ee60 Doc fixes. 2023-07-18 19:55:46 +04:00
Kevin Lloyd Bernal 3f5bbe3a8f introduce CLOSESPIDER_TIMEOUT_NO_ITEM in CloseSpider 2023-07-18 18:30:21 +08:00
Andrey Rakhmatullin cdda8ad46d Add docs about fallbacks in addons. 2023-07-17 21:05:58 +04:00
Andrey Rakhmatullin a2264d3b8b Improve docs about setting settings in addons. 2023-07-14 18:57:27 +04:00
Andrey Rakhmatullin c92c9af075 Add create_instance support to addons. 2023-06-29 15:04:46 +04:00
Andrey Rakhmatullin 9612ae3e93 Remove more code. 2023-06-28 18:44:30 +04:00
Andrey Rakhmatullin e7124447f7 Remove unneeded code. 2023-06-26 16:57:46 +04:00
Andrey Rakhmatullin 4e21e3e59d Merge remote-tracking branch 'origin/master' into addons 2023-06-26 14:25:35 +04:00
Alex 04ee3303e4
Adding support for Windows of absolute pathlib.Path objects in FeedExporter (#5939) 2023-06-22 07:04:06 +02:00
Aaron Smith ee215a2970
Change redirect text from Response.request docs (#5937) 2023-06-21 20:05:39 +02:00
Laerte Pereira 0097b4c0bb cleanup: Remove `pkg_resources` usage 2023-06-20 16:40:38 -03:00
Serhii A 777a6ea412
Make the retry middleware exception list configurable (#5929) 2023-06-16 15:46:06 +02:00
Andrey Rakhmatullin f582246d7b More doc fixes. 2023-06-15 18:49:08 +04:00
Andrey Rakhmatullin 2f9ebb66c3 Remove some dead code/docs. 2023-06-15 17:00:38 +04:00
Andrey Rakhmatullin 27f5f35134 More quick doc fixes. 2023-06-14 18:30:33 +04:00
Andrey Rakhmatullin fdfab17438 Fix docs for the renamed ADDONS setting. 2023-06-14 18:00:18 +04:00
Andrey Rakhmatullin 22bd0d9a79 Fix docs build. 2023-06-14 17:58:11 +04:00
Andrey Rakhmatullin 7ebb8256f0 Some cleanup. 2023-06-14 17:52:11 +04:00
Andrey Rakhmatullin 075ad6f196 Test, linter etc. fixes. 2023-06-14 16:34:18 +04:00
Andrey Rakhmatullin 898e3045a1 Merge branch 'master' into addons 2023-06-14 15:37:08 +04:00
Andrey Rakhmatullin 85fe88f80f
Merge pull request #5833 from GodelTech/issue_960_s3_feed_export 
Fix #960: S3 Feed Export throws boto error
 2023-06-13 18:44:06 +04:00
Andrey Rakhmatullin 8a64f3e8de
Merge branch 'master' into feat/FixFeedExport 2023-06-13 18:21:06 +04:00
Andrey Rakhmatullin 33b418dc84
Merge pull request #5927 from bulatbulat48/master 
Add additional requests examples.
 2023-05-16 19:13:02 +04:00
Adrián Chaves d362699fa3 Move inline request examples to the coroutines documentation 2023-05-16 13:39:02 +02:00
Adrián Chaves e4cf8fc121
Update asyncio.rst 2023-05-15 18:51:58 +02:00
isabela_catanante 5adada5d19 Improve the overwrite feed option documentation 2023-05-12 12:55:24 +02:00
bulat 8de2064ba3 add import. 2023-05-11 13:22:33 +05:00
bulat 4878cc7ef0 Add proper imports. 2023-05-11 13:19:40 +05:00
bulat b62c1263de add import to the example. 2023-05-11 13:15:30 +05:00
bulat fc2d1b2171 make example reachable. 2023-05-11 12:56:29 +05:00
bulat 6a0bbad677 Merge branch 'master' of https://github.com/bulatbulat48/scrapy 2023-05-11 12:54:04 +05:00
bulat 6194db1335 Update title. 2023-05-11 12:54:01 +05:00
Bulat Khabibullin 85103b4932
add proper example 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-05-11 12:53:43 +05:00
Bulat Khabibullin 26374e21f8
Update docs/topics/asyncio.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-05-10 18:32:36 +05:00
Bulat Khabibullin 57f3140daa
Update docs/topics/asyncio.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-05-10 18:31:54 +05:00
bulat 87d10161cd Add selector as class. 2023-05-10 15:35:48 +05:00
bulat b1f4017788 Refactor batch sample. 2023-05-10 15:34:58 +05:00
bulat 99b0ece165 remove extra line. 2023-05-09 20:27:46 +05:00
bulat d32c678234 Update description. 2023-05-09 19:02:34 +05:00
bulat 2c6a772145 Merge branch 'master' of https://github.com/bulatbulat48/scrapy 2023-05-09 18:57:45 +05:00
bulat a75231a1ec fix underline. 2023-05-09 18:57:43 +05:00
Bulat Khabibullin 75cd4958fe
Merge branch 'scrapy:master' into master 2023-05-09 18:05:40 +05:00
bulat c327a92e97 add additional requests examples. 2023-05-09 18:04:18 +05:00
Andrey Rakhmatullin 5bb6dfbc25
Merge pull request #5909 from scrapy/relnotes-2.9 
Release notes for 2.9.0
 2023-05-08 14:27:21 +04:00
Andrey Rakhmatullin cba891a66c Enable doc tests for selectors.rst, fix issues. 2023-05-04 15:04:33 +04:00
Andrey Rakhmatullin ea15ff1d32
Merge pull request #5915 from jxlil/feature/support-parsel-jmespath 
Added support for the Parsel JMESPath feature
 2023-05-03 13:18:39 +04:00
Jalil SA 865c36bdbb update docs 2023-04-28 08:56:11 -06:00
guillermo-bondonno b50c032ee9
Add feed_slot_closed and feed_exporter_closed signals (#5876) 2023-04-26 08:20:37 +02:00
Andrey Rakhmatullin 67bfb304cd Release notes for the current master. 2023-04-21 20:49:51 +04:00
Andrey Rakhmatullin 1e20ba0a1b
Merge branch 'master' into feat/FixFeedExport 2023-04-11 19:48:19 +04:00
Mojtaba Dashtinejad 98571eb946
Add missing slash in selecting dynamically-loaded content documentation (#5890) 2023-04-10 14:14:49 +02:00
Andrey Rakhmatullin 96033ce5a7
Merge pull request #5883 from sdidier-dev/patch-1 
Add missing 'crawl' command in crawl examples ✏️
 2023-03-31 14:47:52 +04:00
Sébastien DIDIER 6d94aa061c
Add missing 'crawl' command in crawl examples 2023-03-31 11:05:20 +02:00
Jalil SA 44cdaa442b
Update docs/topics/spiders.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-03-17 13:19:03 -06:00
Jalil SA 24f28c415c fix: docs update_settings() 2023-03-17 12:16:08 -06:00
Jalil SA a1fc37cbff
Update docs/topics/spiders.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-03-17 12:13:05 -06:00
Jalil SA 495372648c fix: docs update_settings() 2023-03-16 23:14:57 -06:00
jazzthief cb67bc17b7
Remove `botocore` from docs 2023-03-16 17:25:05 +01:00
namelessGonbai 6ab49e954f
Update docs/topics/feed-exports.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-03-16 18:03:06 +09:00
kenshi kikuchi 50801c7207 Fix Docs 2023-03-16 17:17:20 +09:00
Adrián Chaves dfbb63a2f1 Merge remote-tracking branch 'scrapy/master' into issue_960_s3_feed_export 2023-03-15 14:07:54 +01:00
Alex 101a0c32d7
Support genspider with HTTPS (#5808) 2023-03-14 08:13:44 +01:00
Andrey Rakhmatullin 3659a8c8d9
Merge pull request #5328 from GeorgeA92/per_slot_settings 
Per slot settings
 2023-03-09 22:31:24 +05:00
Jalil SA 96d51c3afa docs: update 2023-03-08 04:21:21 -06:00
Jalil SA 1d862d0831 fix: remove line breaks 2023-03-08 03:26:38 -06:00
Jalil SA 05893e1796 docs: Spider.update_settings 2023-03-08 02:59:47 -06:00
Georgiy Zatserklianyi 218829b1db per slot settings: typo fixed 2023-03-06 16:25:49 +02:00
Georgiy Zatserklianyi be52fe4f67 Revert "periodic log stats: typo in docs fixed" 
This reverts commit 68ba25cb69.
 2023-03-06 16:24:28 +02:00
Georgiy Zatserklianyi 68ba25cb69 periodic log stats: typo in docs fixed 2023-03-04 22:15:06 +02:00
Oleg 2b3a8f0d69
Fix a typo in the LOG_FORMAT description (#5839) 2023-03-02 10:19:41 +01:00
Georgiy Zatserklianyi 045092e8d7 per_slot_settings(docs): code style fix (from previous pre-commit check) 2023-02-16 17:55:59 +02:00
Georgiy Zatserklianyi 07e1429877 per_slot_settings(docs): docs `:setting` bindings added 2023-02-16 15:14:01 +02:00
Yegor 29c2477f0a
Document the need to install boto3 2023-02-15 16:40:05 +01:00
Andrey Rakhmatullin 76eba9977b
Skip or fix failing code block tests (#5826) 2023-02-14 09:40:38 +01:00
Georgiy Zatserklianyi 474087be6f per_slot_settings(docs): settings 3 2023-02-09 14:52:28 +02:00
Georgiy Zatserklianyi 5208d436ae per_slot_settings(docs): settings 2 2023-02-09 00:33:21 +02:00
Georgiy Zatserklianyi c3033a54b1 per_slot_settings(docs): settings 1 2023-02-08 23:55:07 +02:00
pankaj1707k 03f32c018f
Update black version and fix minor typos and punctuations 2023-02-03 11:52:20 +05:30
Pankaj Kumar 280cd6ce71
Merge branch 'scrapy:master' into blacken-docs 2023-02-03 11:38:26 +05:30
pankaj1707k 7fdbbd3ccb
Fix for python 3.11 (docs) check on pull request 2023-02-02 15:52:22 +05:30
Adrián Chaves 068af85722
Add release notes for Scrapy 2.8 (#5799) 2023-02-02 04:54:02 +01:00
pankaj1707k cc9eb3fa79
Format python console blocks using blacken-docs 
Prepend '.. code-block:: pycon' to make python console blocks detectable by blacken-docs
 2023-02-01 20:37:39 +05:30
pankaj1707k c1bbb299d7
Add and run pre-commit hook 'blacken-docs' 
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
 2023-02-01 16:30:57 +05:30
Adrián Chaves 9272c4af0c Merge remote-tracking branch 'scrapy/master' into no-callback 2023-01-25 17:41:41 +01:00