RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,622 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

3622 Commits

Author SHA1 Message Date
nramirezuy 332bf3b6b7 defaultheaders multi spider support removed 2013-10-09 13:50:15 -02:00
Pablo Hoffman 7ec01799d5 Merge pull request #419 from nramirezuy/robotstxt-multispider 
robotstxt mid multi spider support removed
 2013-10-09 07:43:11 -07:00
Pablo Hoffman 9ad736d7ce Merge pull request #420 from nramirezuy/httpauth-multispider 
httpauth mid multispider support removed
 2013-10-08 21:06:49 -07:00
Pablo Hoffman 27d3cbb64f Merge pull request #421 from nramirezuy/useragent-multispider 
useragent mid multi spider support removed
 2013-10-08 21:06:13 -07:00
Pablo Hoffman 37b01a229c Merge pull request #422 from nramirezuy/downloadtimeout-multispider 
downloadtimeout mid multi spider support removed
 2013-10-08 21:05:06 -07:00
nramirezuy 1d09b6e1f2 downloadtimeout mid multi spider support removed 2013-10-08 19:03:26 -02:00
nramirezuy fbb0ad6419 useragent mid multi spider support removed 2013-10-08 18:18:49 -02:00
nramirezuy f56f547738 httpauth mid multispider support removed 2013-10-08 18:00:25 -02:00
nramirezuy 10646f1768 robotstxt mid multi spider support removed 2013-10-08 16:38:08 -02:00
Pablo Hoffman 3501c2025e remove no longer existent examples from doc_files used in bdist_rpm. closes GH-417 2013-10-08 15:18:45 -02:00
Pablo Hoffman 8b9526a8f6 Merge pull request #400 from irgmedeiros/patch-2 
Update the second code example
 2013-10-07 07:57:18 -07:00
Pablo Hoffman 86c6e9433f remove minor reference to 'scrapy server' command 2013-10-04 14:37:55 -03:00
Daniel Graña aad90ec5a2 Add 0.18.3 release notes 
Conflicts:
	docs/news.rst
 2013-10-03 12:56:25 -03:00
Daniel Graña 479b4942bd fix regression on lazy evaluation of start requests 2013-10-03 12:49:16 -03:00
Pablo Hoffman 662eac11fa added pypi version badge to README 2013-10-03 12:47:22 -03:00
Pablo Hoffman ba7b75d299 Merge pull request #410 from duendex/httpsMockServer 
Adds HTTPS support to the MockServer.
 2013-10-02 14:05:44 -07:00
duendex 2519b3031e Adds HTTPS support to the MockServer. 2013-10-02 17:35:28 -03:00
Daniel Graña 0c63c9539d Merge pull request #409 from kalessin/imagesfield 
allow to set source and destination fields in files and images pipelines, and added tests
 2013-10-02 13:31:26 -07:00
Martin Olveyra e81120e683 allow to set source and destination fields in files and images pipelines, and added tests 2013-10-02 18:23:13 -02:00
Pablo Hoffman 37c24e01d7 document bindaddress request meta 2013-10-02 17:13:17 -03:00
Pablo Hoffman a9c3519897 updated required twisted version to 10.0 2013-10-01 14:07:38 -03:00
Pablo Hoffman fe9ae1d46a Merge pull request #406 from nopper/master 
Handling inconsistencies among Twisted releases
 2013-10-01 09:59:00 -07:00
Francesco Piccinno cc8a93f185 Handling inconsistencies among Twisted releases 2013-10-01 14:36:44 +02:00
Pablo Hoffman 56275b1b9c Merge pull request #404 from darkrho/djangoitem-docs 
Add a section to DjangoItem docs page regarding setting up Django's settings.
 2013-09-30 11:15:32 -07:00
Rolando Espinoza d6e3eae527 docs: added section regarding setting up django's settings. 2013-09-30 09:58:10 -04:00
Rolando Espinoza 0cc1d870db docs: minor tidy up sample code and missing shell prompts. 2013-09-30 09:58:10 -04:00
Mikhail Korobov 36cb54d302 Merge pull request #402 from LorenDavie/master 
Corrected typo.
 2013-09-29 14:21:04 -07:00
Loren Davie 8af0e89e85 Corrected typo. 2013-09-29 17:06:46 -04:00
Pablo Hoffman 1a90ee51c0 Merge pull request #401 from LorenDavie/master 
Added dynamic creation of item classes to practices.rst. Issue
 2013-09-29 07:20:52 -07:00
Loren Davie f49f5724d5 Added dynamic creation of item classes to practices.rst. 2013-09-28 09:00:48 -04:00
irgmedeiros 9b50409986 Update the second code example 
Update the second code example to reflect the last change in the first example.
 2013-09-27 18:22:33 -03:00
Pablo Hoffman 780ef128d2 Merge pull request #399 from irgmedeiros/patch-1 
Update practices.rst
 2013-09-27 14:03:51 -07:00
irgmedeiros d9e0fdc9aa Update practices.rst 
With this modification scrapy runs the spider with project settings. The previous example ran only with default settings resulting in ignoring all user settings as pipelines for example.
 2013-09-27 17:56:30 -03:00
Daniel Graña 265910aae6 Merge pull request #363 from taikano/sitemap_alternate 
also fetch alternate URLs from sitemaps, see #360
 2013-09-26 09:15:02 -07:00
Pablo Hoffman 12280c2a95 fix sphinx references in doc 2013-09-25 15:13:17 -03:00
Mikhail Korobov b74e5aa551 fix ITEM_PIPELINES setting handling 
* item_pipelines variable was unused so the fallback didn't work;
* added support for fallback in case of ITEM_PIPELINIES defined as set.
 2013-09-25 20:37:06 +06:00
Daniel Graña 61e89d8247 Merge pull request #393 from scrapy/itempipe-dict 
Make ITEM_PIPELINE setting a dict
 2013-09-23 14:12:26 -07:00
Pablo Hoffman fc388f4636 Make ITEM_PIPELINE setting a dict 
This is for consistency with how spider and downloader middlewares are
defined. ITEM_PIPELINE_BASE was also added and both remain empty.

Backwards compatibility is kept (with a warning) with list-based
ITEM_PIPELINES.
 2013-09-23 17:50:43 -03:00
Pablo Hoffman b1d1a36a1e add note about enclosing urls with quotes when running from command-line. closes GH-384 2013-09-18 18:01:28 -03:00
Mikhail Korobov bb8cb41777 Merge pull request #391 from amferraz/patch-2 
Update request-response.rst
 2013-09-18 07:59:46 -07:00
cacovsky 71b320914a Update request-response.rst 
Fix small doc typo (too many backticks)
 2013-09-18 11:45:25 -03:00
Pablo Hoffman ee8464b471 Merge pull request #390 from kmike/mock-in-tox 
mock is required for running tests
 2013-09-17 15:51:54 -07:00
Mikhail Korobov 7ffc786fb6 mock is required for running tests since 45ff6ec28a 
This fixes tox test running. Mock is available at Travis, but there is no harm in listing it explicitly.
 2013-09-18 00:02:23 +06:00
Pablo Hoffman 2a89265b96 Merge pull request #387 from ktharmal/patch-1 
Fixed directory location for dmoz_spider.py file
 2013-09-16 11:36:13 -07:00
Kumara Tharmalingam bbb0603091 Fixed directory location for dmoz_spider.py file 
It should be under 'tutorial/spiders' not 'dmoz/spiders'
 2013-09-15 21:55:52 -07:00
Stefan 855041ebfd added test case for comments, see #363 2013-09-14 12:17:34 +02:00
Daniel Graña 0400b18efa docs: list lxml as installation prerequisite 2013-09-09 12:44:26 -03:00
Stefan 6994959181 renamed to sitemap_alternate_links and added default value, see #360 2013-09-08 10:38:28 +02:00
Daniel Graña b326b87154 forms: do not submit reset inputs 2013-09-07 21:53:27 -03:00
Stefan 8ed2d0cda1 improved changes to allow retrieval of alternate links in sitemaps, see #360 2013-09-07 12:56:30 +02:00