RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,726 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

9726 Commits

Author SHA1 Message Date
Adrián Chaves 3f0c2fae5e Skip test_skip_bad_links based on the w3lib version 2023-04-12 09:28:28 +02:00
Samuel Bartlett 618e82dbe1 Exclude test for python versionbs less than 3.8 2023-03-31 15:12:47 +00:00
Samuel Bartlett 608b7de582 Skip new test if python version less than 3.8 2023-03-31 14:38:06 +00:00
Samuel Bartlett c9a5934494 Reduce logging level of bad URL 2023-03-31 12:29:49 +00:00
Samuel Bartlett 4043560547 remove utf-8 encoding flag from test 2023-03-31 12:29:22 +00:00
Samuel Bartlett 00d93026c8 Fix bad test case 2023-03-31 08:30:19 +00:00
Samuel Bartlett 7cb7cf1ad1 Add link extractor back to test 2023-03-31 08:09:02 +00:00
Samuel Bartlett 9cbcf7724d Add test to make sure spider doesn't crash on bad 2023-03-31 08:07:43 +00:00
Samuel Bartlett 9ef00c5c0b Add logging Lines 
Adds an error loggign line to the LinkExtractor to detail encountered
bad links
 2023-03-31 08:01:54 +00:00
Samuel Bartlett 90ce6589ee Add try/except to safe_url_string() 
Added a try catch condition to the safe_url_string() processing
in the LxmlParserLinkExtractor class to avoid scrapers crashing
unneccessarily
 2023-03-30 13:07:51 +00:00
Andrey Rakhmatullin 8045d7eaa5
Merge pull request #5851 from felipeboffnunes/extend_contributing 
#5109 extend contributing.rst
 2023-03-29 16:27:41 +04:00
Andrey Rakhmatullin 4249fc64d1
Merge pull request #5879 from xt1033/bugfix/updated-Makefile-to-enable-string-concatenation 
Fixed Docs Makefile to open build/html/index.html in browser(#5878)
 2023-03-29 12:06:07 +04:00
karza_abhishek 46bb7b31d1 Fixed Docs Makefile to open build/html/index.html in browser(#5878) 2023-03-28 23:23:32 +05:30
Andrey Rakhmatullin 4dacad0d8f
Merge pull request #5877 from jxlil/master 
Fix: `Uncaught ReferenceError: analytics is not defined`
 2023-03-28 14:05:47 +04:00
Jalil SA 2c31aa6c85 delete _template/layout.html 2023-03-28 01:12:19 -06:00
Jalil SA c22c7bd82b fix: rollback GA code 2023-03-27 20:41:19 -06:00
Felipe Boff Nunes af730df83c adjustments 2023-03-27 08:40:17 -03:00
Andrey Rakhmatullin ada9173078
Don't call SSL_get_server_tmp_key() if not available (#5858) 2023-03-27 12:44:44 +02:00
felipeboffnunes 7bcbfabdbc forgot header bind 2023-03-16 10:28:12 -03:00
felipeboffnunes a81fb5002b adjustments, lean approach 2023-03-16 10:23:06 -03:00
felipeboffnunes 3e59b0805e #5109 extend contributing.rst 2023-03-16 10:02:30 -03:00
auxsvr d60b4edd11
Prevent an edge case that creates an extra event loop (#5832) 2023-03-14 09:31:13 +01:00
Alex 101a0c32d7
Support genspider with HTTPS (#5808) 2023-03-14 08:13:44 +01:00
Hugo van Kemenade 9411cf4e70
Replace deprecated BadZipfile with BadZipFile (#5849) 2023-03-13 19:03:41 +01:00
Mohammadtaher Abbasi afafc2781a
Fix parse command issues with asyncio (#5824) 2023-03-10 16:20:15 +01:00
Andrey Rakhmatullin 3659a8c8d9
Merge pull request #5328 from GeorgeA92/per_slot_settings 
Per slot settings
 2023-03-09 22:31:24 +05:00
Jalil SA 8aca47e25d
Add setdefault to BaseSettings (#5821) 2023-03-07 10:26:59 +01:00
Georgiy Zatserklianyi 218829b1db per slot settings: typo fixed 2023-03-06 16:25:49 +02:00
Georgiy Zatserklianyi be52fe4f67 Revert "periodic log stats: typo in docs fixed" 
This reverts commit 68ba25cb69.
 2023-03-06 16:24:28 +02:00
Georgiy Zatserklianyi 68ba25cb69 periodic log stats: typo in docs fixed 2023-03-04 22:15:06 +02:00
Andrey Rakhmatullin 88eec7a417
Merge pull request #5801 from alexpdev/path_object_error_#5739 
Support Path Objects Issue #5739
 2023-03-04 22:01:24 +05:00
Alex 59f5250e59
Merge branch 'scrapy:master' into path_object_error_#5739 2023-03-03 16:47:08 -08:00
Oleg 2b3a8f0d69
Fix a typo in the LOG_FORMAT description (#5839) 2023-03-02 10:19:41 +01:00
Alex 78269f9a4f
Merge branch 'scrapy:master' into path_object_error_#5739 2023-02-25 20:49:04 -08:00
Andrey Rakhmatullin 8fbebfa943
Merge pull request #5805 from scrapy/mypy-imports 
Handle typing issues hidden by follow_imports=skip
 2023-02-24 14:57:59 +05:00
Andrey Rakhmatullin 2970b35cff Merge remote-tracking branch 'origin/master' into mypy-imports 2023-02-24 02:35:09 +04:00
Andrey Rakhmatullin 3dd9d71c32 Correctly wrap connection errors into ResponseFailed. 2023-02-24 01:13:48 +04:00
Andrey Rakhmatullin d20f278882 Add a link to the w3lib typing fix. 2023-02-24 00:28:24 +04:00
Andrey Rakhmatullin d7bf39ee78 Fix a merge problem. 2023-02-24 00:28:06 +04:00
Andrey Rakhmatullin 3a40c06ed9 Bump mypy and types-*. 2023-02-24 00:16:49 +04:00
Andrey Rakhmatullin 733309affa Don't assume formnumber can be None. 2023-02-24 00:05:59 +04:00
Alex 7764184f4b
Merge branch 'scrapy:master' into path_object_error_#5739 2023-02-20 00:15:34 -08:00
Georgiy Zatserklianyi 864eee66c7 per_slot_settings(docs): code style fix 3 (from previous pre-commit check) 2023-02-16 18:19:36 +02:00
Georgiy Zatserklianyi dd5524eb98 per_slot_settings(docs): code style fix 2 (from previous pre-commit check) 2023-02-16 17:59:36 +02:00
Georgiy Zatserklianyi 045092e8d7 per_slot_settings(docs): code style fix (from previous pre-commit check) 2023-02-16 17:55:59 +02:00
Georgiy Zatserklianyi 07e1429877 per_slot_settings(docs): docs `:setting` bindings added 2023-02-16 15:14:01 +02:00
Georgiy Zatserklianyi 98a5958687 per_slot_settings(docs): per slot settings - the highest priority 2023-02-16 14:52:31 +02:00
Andrey Rakhmatullin c8547b0370
Merge pull request #5820 from alexpdev/coveragerc_disable_warnings 
added disable_warnings instruction to .coveragerc
 2023-02-15 03:28:35 +05:00
Alex d366782187
Merge branch 'scrapy:master' into coveragerc_disable_warnings 2023-02-14 05:17:10 -08:00
Alex c5fc4bbb1e
Merge branch 'scrapy:master' into path_object_error_#5739 2023-02-14 05:16:23 -08:00