RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,836 Commits 49 Branches 132 Tags 132 MiB
Commit Graph

6836 Commits

Author SHA1 Message Date
Daniel Graña 76433feb4f
Merge pull request #3548 from scrapy/fix-aws-settings-docs 
fix docs for AWS_... settings. A follow-up to GH-2609.
 2018-12-26 11:10:57 -03:00
Daniel Graña 5cf26c9985
Merge pull request #3547 from scrapy/setup.py-3.7 
declare Python 3.7 support in setup.py
 2018-12-26 11:09:07 -03:00
Daniel Graña d473be2040
Merge pull request #3519 from fpghost/master 
[MRG+1] the strip() isnt needed after base64 encoding user pass
 2018-12-26 11:08:49 -03:00
Daniel Graña 5d02aab9f9
Merge pull request #3496 from frederik-elwert/patch-1 
[MRG+1] Add documentation to `scrapy shell` command.
 2018-12-26 11:04:30 -03:00
Daniel Graña 6a0ea0cf26
Merge pull request #3415 from scrapy/telnet-auth 
[MRG+1] Telnet console authentication
 2018-12-26 10:58:44 -03:00
Mikhail Korobov 71e47629b1 DOC fix docs for AWS_... settings. A follow-up to GH-2609. 2018-12-26 16:35:05 +05:00
Mikhail Korobov 4306886ac6
Merge pull request #3527 from hsiaoyi0504/patch-1 
unify the quote style
 2018-12-26 14:45:06 +05:00
Mikhail Korobov cdd04dfb1d declare Python 3.7 support in setup.py 2018-12-26 13:13:49 +05:00
Mikhail Korobov 7c26701012 DOC warn about telnet console being insecure 2018-12-26 01:33:58 +05:00
Mikhail Korobov dbfabf02e8 Merge branch 'master' into telnet-auth 2018-12-26 01:04:34 +05:00
Daniel Graña 23bffff567
Merge pull request #3526 from lucywang000/update-travis 
remove "sudo: false" now that travis no longer supports it
 2018-12-20 19:54:34 -03:00
Daniel Graña d4fd1236b7
Merge pull request #3538 from scrapy/boto-import-error-under-jessie 
Fix boto import error under Jessie testing environment
 2018-12-20 19:49:55 -03:00
Daniel Graña 8ed6beb7f9 Needs to be installed within tox env 2018-12-20 19:39:29 -03:00
Daniel Graña f6dfc5f3dd Fix boto import error under Jessie testing environment 2018-12-20 19:23:23 -03:00
Daniel Graña 665c04b0c1
Merge pull request #3518 from Gallaecio/scrapy-project-doc 
Document the SCRAPY_PROJECT environment variable
 2018-12-12 11:56:47 -03:00
hsiao yi cd9d8e28cd
unify the quote style 2018-12-11 19:21:07 +08:00
Lucy Wang 4d48759978 remove "sudo: false" now that travis no longer supports it 
https://changelog.travis-ci.com/deprecation-container-based-linux-build-environment-82037
 2018-12-10 14:44:17 +08:00
Mikhail Korobov e766bde3a3
Merge pull request #3517 from Gallaecio/install-troubleshooting 
[MRG+1] Add a troubleshooting section to the installation instructions
 2018-12-06 18:07:43 +05:00
fpghost d7c8eee2fc the strip() isnt needed 2018-12-04 10:57:51 +01:00
Adrián Chaves 62f3349c1a Document the SCRAPY_PROJECT environment variable 
Fixes #1109
 2018-12-03 17:14:10 +01:00
Adrián Chaves 274b65dff4 Add a troubleshooting section to the installation instructions 
Its initial content covers the workaround for #2473.
 2018-12-03 16:36:05 +01:00
Konstantin Lopuhin 886513c375
Merge pull request #3495 from toddrme2178/patch-1 
[MRG+1] Include additional files in sdists
 2018-11-20 11:10:32 +03:00
Frederik Elwert 127bf499f1
Add documentation to `scrapy shell` command. 
The special syntax required for local files (`./file.html`) is not documented as part of the `scrapy shell --help` output. This patch adds that.
 2018-11-16 22:15:03 +01:00
Todd 491929c212
Include additional files in sdists 
In particular this includes files needed for running the tests, as well as the changelog.
 2018-11-16 13:38:19 -05:00
Mikhail Korobov dc65e7527d
Merge pull request #3468 from ilhaoni/feature/update-docs-intro-tutorial 
[MRG+1] Update Scrapy Tutorial docs
 2018-11-06 03:33:09 +05:00
Immanuella Lim 6c98010f11 Remove 'Dive into Python3' reference 2018-11-04 16:04:45 +08:00
Immanuella Lim c9b5bd6ad7 Remove ad link Dive Into Python3 from tutorial docs 2018-10-18 02:22:32 +08:00
Daniel Graña 44f8e28b3c Fix headings' underlines 2018-10-16 19:53:20 -03:00
Daniel Graña 06f2db7fd1
Merge pull request #3445 from jfflisikowski/3097-item-urls-not-defined-in-example 
[MRG+1] Updating debug example file (#3097)
 2018-10-16 19:51:32 -03:00
Daniel Graña 553b3a1c12
Merge pull request #3465 from hcoura/telnet-auth 
Add Telnet console authentication docs
 2018-10-16 15:08:34 -03:00
Henrique Coura 92b7955d75 Add Telnet console authentication docs 2018-10-16 14:50:00 -03:00
jfflisikowski edaf74bfae Correct the unclear comments by adding <# < processing code not shown > 2018-10-02 19:48:48 +02:00
Daniel Graña d80f9ed725
Merge pull request #3429 from hcoura/telnet-auth 
Randomly generate telnet credentials by default
 2018-09-26 14:00:46 -03:00
Henrique Coura 441e1e750f Style changes 2018-09-26 13:28:34 -03:00
Henrique Coura 5f9931d2ad do not log username 2018-09-26 13:07:04 -03:00
Henrique Coura e57a629efc Generate only password, encode username/password only on login 2018-09-26 11:54:57 -03:00
Henrique Coura 37cfb49805 Randomly generate telnet credentials by default 2018-09-24 16:42:49 -03:00
Daniel Graña bafd174a9f
Merge pull request #3390 from scrapy/parsel-1.5 
[MRG+1] update Scrapy to use parsel 1.5
 2018-09-18 13:07:41 -03:00
Mikhail Korobov ffbd33edac DOC mention gotcha with `foo::text` selector and empty `foo` elements 
also, move "Selecting attributes" reference closer to `a::atr(href)` example
 2018-09-18 05:03:35 +05:00
Mikhail Korobov 2c3b2158c9 DOC address @stummjr's review comments 
* fixed several small issues
* re-written "Creating Selectors" section
* fixed remaining .extract usage in tests
 2018-09-18 05:02:17 +05:00
Mikhail Korobov 9db21e5502 DOC fix remove_namespaces example 
See https://github.com/scrapy/parsel/pull/119
 2018-09-15 02:43:37 +05:00
Mikhail Korobov dc95ecbe25 DOC use autodocs for selectors; document more methods and attributes; suggest get/getall 2018-09-12 18:36:25 +05:00
Mikhail Korobov 7fdfdb7fa2 DOC reorganize selectors tutorial, port more topics from parsel docs, adjust wording in the introduction 2018-09-12 17:57:27 +05:00
Mikhail Korobov bdcc045f62 DOC switch from .extract to get/getall API in docs 
Also, response.urljoin is added in a few places, for robustness.
 2018-09-12 17:57:27 +05:00
Mikhail Korobov afce9716fa DOC mention .attrib in the tutorial 2018-09-12 17:57:27 +05:00
Mikhail Korobov 12e42bbe06 switch SgmlLinkExtractor to .getall 2018-09-12 17:57:27 +05:00
Mikhail Korobov 460f0f0451 [backwards incompatible] switch ItemLoader from .extract to .getall. 
This change is backwards incompatible if ItemLoader is used with a custom Selector
subclass which overrides .extract without overriding .getall.
 2018-09-12 17:57:27 +05:00
Mikhail Korobov 8c29be606c update spider templates to use .get 2018-09-12 17:57:27 +05:00
Mikhail Korobov 53da56c8dc TST update tests to use get/getall/attrib instead of extract 2018-09-12 17:57:27 +05:00
Mikhail Korobov 2c48d156db DOC cleanup references in tutorials: 
* remove unused link
* fix ReST syntax
* fix a link to regular expression docs
 2018-09-12 17:57:27 +05:00