RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,222 Commits 42 Branches 132 Tags 161 MiB
Commit Graph

6222 Commits

Author SHA1 Message Date
Paul Tremberth fbb5559299 Add tests for crawl command non-default cases 2016-09-12 13:35:14 +02:00
Andrew Hlynskyi 80260824c6 Fix completion in `scrapy shell` for new imports 2016-09-12 01:13:23 +03:00
Joakim Uddholm 743a0aa422 Two fixes for when using the parse command and the '-r' flag (rules). 
1. Use default "parse" as callback when the matching rule has no callback.
2. Log error and return when no rule matches the parsed url.
 2016-09-08 21:52:14 +02:00
Matti Remes 0ef570f6f0 Update exceptions.rst 
Added the missing dot. (+1 squashed commit)
Squashed commits:
[2198972] Update exceptions.rst

There are namely no constructors in classes in Python but an ``__init__`` method instead.
 2016-09-08 19:38:17 +05:00
Paul Tremberth ec4ab126b6 Merge pull request #2220 from scrapy/comment-typo-fix 
typo fix in HttpProxyMiddleware
 2016-09-07 10:15:12 +02:00
Mikhail Korobov 960b1bc8f0 typo fix in HttpProxyMiddleware 2016-09-07 04:54:32 +05:00
Paul Tremberth 9e6a72cc4b Merge pull request #2217 from stummjr/add-analytics-to-docs 
[MRG+1] Add Segment Analytics to Documentation
 2016-09-05 15:59:58 +02:00
Valdir Stumm Junior 9cea6f0730 Add Segment Analytics to Documentation 2016-09-02 14:51:07 -03:00
Paul Tremberth b188f61b95 Update release notes for upcoming 1.2.0 version 2016-09-01 17:38:38 +02:00
Paul Tremberth 58cd7bf895 Remove "precise" test env from Travis-CI config 2016-09-01 11:17:53 +02:00
Paul Tremberth 2b2bfcea88 Add "jessie" build to Travis-CI config 2016-09-01 10:20:49 +02:00
Paul Tremberth 22e870e955 Add Debian Jessie test env 2016-09-01 10:19:49 +02:00
Paul Tremberth eedb6ce774 Merge pull request #2190 from stummjr/fix-docs 
[MRG+1] Fix RANDOMIZE_DOWNLOAD_DELAY description in the docs
 2016-08-31 11:51:47 +02:00
Elias Dorneles 1e95bf59ef Merge pull request #2197 from thomdixon/improve-response-documentation 
[MRG+1] Correct documentation about Response parameters
 2016-08-29 11:08:30 -03:00
Mikhail Korobov 495d322691 DOC move Data Flow below the picture; add links to components 2016-08-26 20:16:22 +05:00
Thom Dixon f68dc3026d Fix indentation 2016-08-24 09:11:27 -07:00
Thom Dixon 633abfbea1 Correct documentation about Response parameters 
This fixes issue #2196
 2016-08-24 08:47:52 -07:00
Valdir Stumm Junior d61650d843 fix RANDOMIZE_DOWNLOAD_DELAY description in the docs 2016-08-19 18:24:32 -03:00
Paul Tremberth cacd038b10 Merge pull request #2188 from scrapy/release-notes-1.1.2-master 
Add release notes for 1.1.2 version
 2016-08-19 17:04:28 +02:00
Paul Tremberth f18c3e5ce5 Add release notes for 1.1.2 version 2016-08-19 17:01:57 +02:00
Paul Tremberth 9de6f1ca75 Merge pull request #1905 from rootAvish/duplication-fix 
[MRG+1] Modified read failure recovery in utils/gz.py to read only the last f.extrasize bytes of f.extrabuf[ ]
 2016-08-17 14:51:30 +02:00
Mikhail Korobov 241bd00e76 Merge pull request #2168 from advarisk/w3lib-canonicalize-url 
[MRG+1] Use w3lib.url.canonicalize_url() from w3lib 1.15.0
 2016-08-16 20:59:17 +06:00
Ashish Kulkarni bb3b806467 Use w3lib.url.canonicalize_url() from w3lib 1.15.0 
Also remove code/imports which are now unused due to this change.

fixes #2157
 2016-08-16 17:42:16 +05:30
Paul Tremberth 9a734e6759 Merge pull request #2058 from dalleng/serialize_set 
[MRG+1] Add set serialization to ScrapyJSONEncoder
 2016-08-12 18:28:34 +02:00
rootavish d9437fd3d9 Modifying existing gzip read failure recovery mechanism to patch read for broken archives 2016-08-11 18:21:42 +05:30
Paul Tremberth 1ec210068f Merge pull request #2169 from Tethik/parse_command_callback_typo 
[MRG+1] Typo fix for error in parse command
 2016-08-11 11:52:59 +02:00
Joakim Uddholm 625c69fdc7 Fixed typo in error message when selecting a callback method for the parse command. 2016-08-08 14:32:53 +02:00
Mikhail Korobov 414857a593 Merge pull request #2140 from jesuslosada/images-expires 
[MRG+1] Fix IMAGES_EXPIRES default value
 2016-08-05 21:52:27 -04:00
Mikhail Korobov fa78849e33 Merge pull request #2165 from loreguerra/master 
[MRG+1] Updated architecture graph for organization/clarity
 2016-08-05 21:46:28 -04:00
Lorena 7d432872bf text updates to match graphic 2016-08-04 11:01:14 -07:00
Lorena 04f93e096c updated graph for organization/clarity 2016-08-04 10:04:47 -07:00
Paul Tremberth 27d4cea6a5 Merge pull request #2161 from redapple/release-notes-1.1.1-master 
Release notes for 1.1.1
 2016-08-01 20:50:14 +02:00
Paul Tremberth 5b1d98b8c8 Update 1.1.1 release date 2016-08-01 20:21:12 +02:00
Paul Tremberth 928e93f8f3 Update notes with latest 1.1 commits 2016-08-01 20:21:12 +02:00
Paul Tremberth e1d118d5ca Update release notes for upcoming 1.1.1 release 2016-08-01 20:21:12 +02:00
Paul Tremberth 2a0a96aef0 Merge pull request #2160 from stummjr/patch-1 
[MRG+1] Remove README download count badge
 2016-08-01 17:57:57 +02:00
Valdir Stumm Jr 63876fc690 Remove download stats badge 2016-08-01 12:16:50 -03:00
Elias Dorneles 4eec05351f remove bumpversion prerelease configuration 
I propose to remove the prerelease configuration from bumpversion,
because I think its behavior is just too confusing.

The rational for this is that making the release procedure predictable
is more important than facilitating making pre-releases, which are sort
of the exception in the workflow.

The current configuration makes most common cases confusing:

* bug fix releases require you have to remember to use `--serialize "{major}.{minor}.{patch}"`
* to start a pre-release cycle, you actually use `minor` or `patch`
* to do the actual minor or patch release, you use `prerel`

Also, `prerel` breaks if you run it on a branch with a final release,
because it can't parse the prerelease information.

Therefore, I propose keeping the bumpversion defaults, and do the
prereleases (dev1, dev2, rc1, etc) manually (with `--new-version`),
which makes for a more predictable and intuitive behavior.

* `bumpversion minor` and `bumpversion patch` will work as expected
* pre-releases will be manually handled, but this seems a small overhead
  than remembering the details I mention above.

If you're happy with this, I'll also update [the wiki][1] with new
instructions.

[1]: https://github.com/scrapy/scrapy/wiki/Scrapy-release-procedure
 2016-08-01 11:37:51 -03:00
Mikhail Korobov 2c9a38d1f5 Merge pull request #2153 from Digenis/Selector_bad_args 
[MRG+1] Selector should not receive both response and text
 2016-07-31 21:28:38 -04:00
Νικόλαος-Διγενής Καραγιάννης 643dbeffcf Selector should not receive both response and text 2016-07-30 10:35:16 +03:00
Elias Dorneles 34e7dadf38 Merge pull request #1610 from darshanime/scheduler_debug 
[MGR+1] Change, document `LOG_UNSERIALIZABLE_REQUESTS`
 2016-07-29 10:12:52 -03:00
darshanime d8e62e660d update log demo print 2016-07-26 20:46:12 +05:30
Paul Tremberth fe68a45c82 Merge pull request #2138 from jesuslosada/master 
[MRG+1] Apply the FILES_STORE_S3_ACL setting in ImagesPipeline
 2016-07-26 12:50:25 +02:00
Elias Dorneles b6375d3701 Merge pull request #1566 from darshanime/signal-example 
[MRG+1] Include example for signal docs
 2016-07-25 10:49:29 -03:00
Darshan Chaudhary a2e6452554 Include signal example 2016-07-25 18:41:25 +05:30
darshanime 0c77b6d033 update docs for settings 2016-07-25 18:05:51 +05:30
Darshan Chaudhary 2d9e5937c6 Include deprecated warning 2016-07-25 18:05:51 +05:30
Darshan Chaudhary 472a8a47d0 Change name, log once 2016-07-25 18:03:43 +05:30
Jesús Losada 8221556089 Add documentation on Amazon S3 ACLs 2016-07-22 23:22:18 +02:00
Diego Allen e17fdd7276 Add set serialization to ScrapyJSONEncoder 2016-07-22 17:20:03 -04:00