RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,710 Commits 50 Branches 132 Tags 142 MiB
Commit Graph

10710 Commits

Author SHA1 Message Date
Adrián Chaves 3f0c2fae5e Skip test_skip_bad_links based on the w3lib version 2023-04-12 09:28:28 +02:00
Kartik Kumar d47c732ae9
base64-decode GCS checksums (#5891) 2023-04-11 18:25:42 +02:00
Andrey Rakhmatullin 1e20ba0a1b
Merge branch 'master' into feat/FixFeedExport 2023-04-11 19:48:19 +04:00
Andrey Rakhmatullin 283e1eb302
Merge pull request #5880 from Prathm-s/Prathms/#1611 
return failure added to _sent_failed
 2023-04-11 17:38:06 +04:00
Andrey Rakhmatullin c7730627a0
Typing improvements for some core components (#5889) 2023-04-11 13:47:41 +02:00
Andrey Rakhmatullin bdb78b9aa5
Merge pull request #5581 from felipeboffnunes/fix_post_processing_with_s3 
Fix Feed Exporter issue with Post Processing
 2023-04-11 15:06:11 +04:00
Felipe Boff Nunes 23017e6e92 adjust 2023-04-10 12:30:00 -03:00
Mojtaba Dashtinejad 98571eb946
Add missing slash in selecting dynamically-loaded content documentation (#5890) 2023-04-10 14:14:49 +02:00
Felipe Boff Nunes a0e2e36b52 adjustments 2023-04-06 14:23:19 -03:00
Felipe Boff Nunes 0ffd1667ba adjustments 2023-04-06 14:17:35 -03:00
Felipe Boff Nunes dead39fd5f
Merge pull request #1 from Gallaecio/fix_post_processing_with_s3 
Test the life cycle of a storage file
 2023-04-06 13:47:34 -03:00
Samuel Bartlett 618e82dbe1 Exclude test for python versionbs less than 3.8 2023-03-31 15:12:47 +00:00
Samuel Bartlett 608b7de582 Skip new test if python version less than 3.8 2023-03-31 14:38:06 +00:00
Samuel Bartlett c9a5934494 Reduce logging level of bad URL 2023-03-31 12:29:49 +00:00
Samuel Bartlett 4043560547 remove utf-8 encoding flag from test 2023-03-31 12:29:22 +00:00
Andrey Rakhmatullin 96033ce5a7
Merge pull request #5883 from sdidier-dev/patch-1 
Add missing 'crawl' command in crawl examples ✏️
 2023-03-31 14:47:52 +04:00
Sébastien DIDIER 6d94aa061c
Add missing 'crawl' command in crawl examples 2023-03-31 11:05:20 +02:00
Samuel Bartlett 00d93026c8 Fix bad test case 2023-03-31 08:30:19 +00:00
Samuel Bartlett 7cb7cf1ad1 Add link extractor back to test 2023-03-31 08:09:02 +00:00
Samuel Bartlett 9cbcf7724d Add test to make sure spider doesn't crash on bad 2023-03-31 08:07:43 +00:00
Samuel Bartlett 9ef00c5c0b Add logging Lines 
Adds an error loggign line to the LinkExtractor to detail encountered
bad links
 2023-03-31 08:01:54 +00:00
Samuel Bartlett 90ce6589ee Add try/except to safe_url_string() 
Added a try catch condition to the safe_url_string() processing
in the LxmlParserLinkExtractor class to avoid scrapers crashing
unneccessarily
 2023-03-30 13:07:51 +00:00
Prathm-s b83fa60a0a return added to _sent_failed 2023-03-30 16:57:56 +05:30
Andrey Rakhmatullin 8045d7eaa5
Merge pull request #5851 from felipeboffnunes/extend_contributing 
#5109 extend contributing.rst
 2023-03-29 16:27:41 +04:00
Andrey Rakhmatullin 4249fc64d1
Merge pull request #5879 from xt1033/bugfix/updated-Makefile-to-enable-string-concatenation 
Fixed Docs Makefile to open build/html/index.html in browser(#5878)
 2023-03-29 12:06:07 +04:00
karza_abhishek 46bb7b31d1 Fixed Docs Makefile to open build/html/index.html in browser(#5878) 2023-03-28 23:23:32 +05:30
Andrey Rakhmatullin 4dacad0d8f
Merge pull request #5877 from jxlil/master 
Fix: `Uncaught ReferenceError: analytics is not defined`
 2023-03-28 14:05:47 +04:00
Jalil SA 2c31aa6c85 delete _template/layout.html 2023-03-28 01:12:19 -06:00
Jalil SA c22c7bd82b fix: rollback GA code 2023-03-27 20:41:19 -06:00
Felipe Boff Nunes af730df83c adjustments 2023-03-27 08:40:17 -03:00
Andrey Rakhmatullin ada9173078
Don't call SSL_get_server_tmp_key() if not available (#5858) 2023-03-27 12:44:44 +02:00
Jalil SA 44cdaa442b
Update docs/topics/spiders.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-03-17 13:19:03 -06:00
Jalil SA 24f28c415c fix: docs update_settings() 2023-03-17 12:16:08 -06:00
Jalil SA a1fc37cbff
Update docs/topics/spiders.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-03-17 12:13:05 -06:00
Jalil SA 495372648c fix: docs update_settings() 2023-03-16 23:14:57 -06:00
jazzthief cb67bc17b7
Remove `botocore` from docs 2023-03-16 17:25:05 +01:00
jazzthief 4ebc08ef10
Switch from `is_boto3_available()` to `IS_BOTO3_AVAILABLE` var 2023-03-16 17:24:11 +01:00
jazzthief a17d996da2
Change `boto3` version to 1.20.0 for `extra-deps-pinned` env 2023-03-16 17:20:22 +01:00
felipeboffnunes 7bcbfabdbc forgot header bind 2023-03-16 10:28:12 -03:00
felipeboffnunes a81fb5002b adjustments, lean approach 2023-03-16 10:23:06 -03:00
felipeboffnunes 3e59b0805e #5109 extend contributing.rst 2023-03-16 10:02:30 -03:00
namelessGonbai 6ab49e954f
Update docs/topics/feed-exports.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-03-16 18:03:06 +09:00
kenshi kikuchi 50801c7207 Fix Docs 2023-03-16 17:17:20 +09:00
kenshi kikuchi c8ed793257 Fix test_export_no_items_multiple_feeds 2023-03-16 17:16:14 +09:00
Yegor Statkevich ffef837a1f
Merge pull request #1 from Gallaecio/issue_960_s3_feed_export 
Adapt tests for a switch to boto3 with botocore support for backward compatibility
 2023-03-15 18:51:37 +03:00
Adrián Chaves 7e7b41c6b3 Fix test expectations for boto3 2023-03-15 16:38:13 +01:00
Adrián Chaves 590955fac8 Provide separate test environments for botocore and boto3 extras; add extra-deps-pinned and remote extras from pinned 2023-03-15 16:03:44 +01:00
Adrián Chaves 39dbfa1d82 Minimize test reliance on S3; do not install botocore on the default test environments 2023-03-15 15:31:39 +01:00
Adrián Chaves dfbb63a2f1 Merge remote-tracking branch 'scrapy/master' into issue_960_s3_feed_export 2023-03-15 14:07:54 +01:00
auxsvr d60b4edd11
Prevent an edge case that creates an extra event loop (#5832) 2023-03-14 09:31:13 +01:00