RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,877 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

1690 Commits

Author SHA1 Message Date
Cyril Mouchel c94cd5b51b
Drop few words about getall selector being called 
To avoid people opening scrapy source code to make sure getall is called over get
 2019-02-17 20:17:50 +01:00
Adrián Chaves 38af090f4d Indicate that users must implement their own authentication result check 
The example of form-based login could lead some users to think its authentication result
check was final. See https://stackoverflow.com/a/54410966/939364

This change should make it more obvious that users are expected to implement their
own logic to check whether authentication worked or not.
 2019-02-04 11:17:58 +01:00
Daniel Graña 65d631329a Be consistent with domain used for links to documentation website 2019-01-31 01:28:53 -03:00
Daniel Graña 88326cd8be Set release date to 1.6.0 2019-01-31 01:16:45 -03:00
Mikhail Korobov 91791cd329 DOC final changelog cleanups 2019-01-30 12:49:36 -03:00
Mikhail Korobov 2c8c8b2dd8 DOC fix after bad merge - remove duplicate entries in changelog 2019-01-30 12:49:36 -03:00
Mikhail Korobov 0fc9d705c2 DOC mention that telnet security improvements happened in 1.5.2 2019-01-30 12:49:36 -03:00
Mikhail Korobov 4cf4dd1d3e DOC add recent changes to changelog 2019-01-30 12:49:36 -03:00
Mikhail Korobov 638469f9ef DOC extract_first/extract matches get/getall better 
Thanks @Gallaecio!
 2019-01-30 12:49:36 -03:00
Mikhail Korobov e479f5aa15 DOC update changelog 
* changes from recently merged pull requests
* more highlights
* re-organized headers
* Selector API changes
 2019-01-30 12:49:36 -03:00
Mikhail Korobov 706910790b [wip] draft 1.6 release notes 2019-01-30 12:49:36 -03:00
Daniel Graña b5026b842c
Merge pull request #3544 from joaquingx/fix-item-pipeline-x 
Fix item-pipeline example
 2019-01-28 10:44:19 -03:00
Harry Moreno 8fca98616a fix grammar 2019-01-28 10:19:31 -03:00
Harry Moreno b828b5f8c8
fix grammar 2019-01-26 18:39:05 -05:00
Daniel Graña 71743a6546 Add release notes for v1.5.2 2019-01-22 18:56:47 -03:00
Joaquin Garmendia Cabrera e1f8b55ba0
Improve syntax for readability 2018-12-28 16:53:12 -05:00
Victor Torres 5a824c906c using shorter import version and moving datetime import to the beginning of the code snippet 2018-12-27 18:34:41 -03:00
Victor Torres bfbcf52e9d fix SitemapSpider import 2018-12-27 18:12:31 -03:00
Victor Torres b68308779a improving docs 2018-12-27 17:37:59 -03:00
Victor Torres e1597f7c42 improve readability 2018-12-27 11:40:14 -03:00
Victor Torres 10f46bca54 documenting sitemap entries as suggested by @kmike 2018-12-27 11:40:14 -03:00
Victor Torres 657f0663b3 rename param from urls to entries 2018-12-27 11:40:14 -03:00
Victor Torres 672385a371 using a method definition instead of a None attribute 2018-12-27 11:40:14 -03:00
Victor Torres a5e1b7bb47 add sitemap_filter attribute to SitemapSpider class 
it makes it possible to filter sitemap urls by any available attribute

for example, you can filter urls with lastmod greater than a given datetime

it can be helpful when the url loc itself does not aggregate that information
 2018-12-27 11:40:14 -03:00
Daniel Graña 76433feb4f
Merge pull request #3548 from scrapy/fix-aws-settings-docs 
fix docs for AWS_... settings. A follow-up to GH-2609.
 2018-12-26 11:10:57 -03:00
Daniel Graña 6a0ea0cf26
Merge pull request #3415 from scrapy/telnet-auth 
[MRG+1] Telnet console authentication
 2018-12-26 10:58:44 -03:00
Mikhail Korobov 71e47629b1 DOC fix docs for AWS_... settings. A follow-up to GH-2609. 2018-12-26 16:35:05 +05:00
Mikhail Korobov 4306886ac6
Merge pull request #3527 from hsiaoyi0504/patch-1 
unify the quote style
 2018-12-26 14:45:06 +05:00
Mikhail Korobov 7c26701012 DOC warn about telnet console being insecure 2018-12-26 01:33:58 +05:00
Mikhail Korobov dbfabf02e8 Merge branch 'master' into telnet-auth 2018-12-26 01:04:34 +05:00
Joaquin Garmendia Cabrera f85c915872
Update item-pipeline example 2018-12-23 00:26:58 -05:00
Daniel Graña 665c04b0c1
Merge pull request #3518 from Gallaecio/scrapy-project-doc 
Document the SCRAPY_PROJECT environment variable
 2018-12-12 11:56:47 -03:00
hsiao yi cd9d8e28cd
unify the quote style 2018-12-11 19:21:07 +08:00
Adrián Chaves 62f3349c1a Document the SCRAPY_PROJECT environment variable 
Fixes #1109
 2018-12-03 17:14:10 +01:00
Adrián Chaves 274b65dff4 Add a troubleshooting section to the installation instructions 
Its initial content covers the workaround for #2473.
 2018-12-03 16:36:05 +01:00
Immanuella Lim 6c98010f11 Remove 'Dive into Python3' reference 2018-11-04 16:04:45 +08:00
Immanuella Lim c9b5bd6ad7 Remove ad link Dive Into Python3 from tutorial docs 2018-10-18 02:22:32 +08:00
Daniel Graña 44f8e28b3c Fix headings' underlines 2018-10-16 19:53:20 -03:00
Henrique Coura 92b7955d75 Add Telnet console authentication docs 2018-10-16 14:50:00 -03:00
jfflisikowski edaf74bfae Correct the unclear comments by adding <# < processing code not shown > 2018-10-02 19:48:48 +02:00
Mikhail Korobov ffbd33edac DOC mention gotcha with `foo::text` selector and empty `foo` elements 
also, move "Selecting attributes" reference closer to `a::atr(href)` example
 2018-09-18 05:03:35 +05:00
Mikhail Korobov 2c3b2158c9 DOC address @stummjr's review comments 
* fixed several small issues
* re-written "Creating Selectors" section
* fixed remaining .extract usage in tests
 2018-09-18 05:02:17 +05:00
Mikhail Korobov 9db21e5502 DOC fix remove_namespaces example 
See https://github.com/scrapy/parsel/pull/119
 2018-09-15 02:43:37 +05:00
Mikhail Korobov dc95ecbe25 DOC use autodocs for selectors; document more methods and attributes; suggest get/getall 2018-09-12 18:36:25 +05:00
Mikhail Korobov 7fdfdb7fa2 DOC reorganize selectors tutorial, port more topics from parsel docs, adjust wording in the introduction 2018-09-12 17:57:27 +05:00
Mikhail Korobov bdcc045f62 DOC switch from .extract to get/getall API in docs 
Also, response.urljoin is added in a few places, for robustness.
 2018-09-12 17:57:27 +05:00
Mikhail Korobov afce9716fa DOC mention .attrib in the tutorial 2018-09-12 17:57:27 +05:00
Mikhail Korobov 2c48d156db DOC cleanup references in tutorials: 
* remove unused link
* fix ReST syntax
* fix a link to regular expression docs
 2018-09-12 17:57:27 +05:00
Mikhail Korobov 09fd6c2a81 DOC unlink Firefox & Firebug sections from the tutorial for now. 
See https://github.com/scrapy/scrapy/issues/3373 and https://github.com/scrapy/scrapy/issues/3372 for motivation.
 2018-09-12 17:57:27 +05:00
Mikhail Korobov d32c4deaa9 DOC update Scrapy selectors tutorial to match parsel's tutorial better 2018-09-12 17:57:27 +05:00