RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,351 Commits 49 Branches 132 Tags 174 MiB
Commit Graph

4351 Commits

Author SHA1 Message Date
andrewshir dfca7b3c80 Fix bug for ".local" host name 
It's necessary to put new list member in squared brackets (i.e. create new list) to merge lists properly, otherwise we will get result list with character elements instead of string element.
 2014-09-06 18:23:27 +06:00
Pablo Hoffman acb0a61cf1 Merge pull request #871 from eltermann/master 
Removed unused 'load=False' parameter from walk_modules()
 2014-09-02 12:55:04 -03:00
eltermann 1dff1fbf75 Removed unused 'load=False' parameter from walk_modules() 2014-09-02 08:33:36 -03:00
Daniel Graña 5daa14770b Merge branch 'Curita-per-spider-settings' 2014-09-01 21:58:13 -03:00
Julia Medina c2592b39fd Test verifying that CrawlerRunner populates spider class settings 2014-09-01 21:56:57 -03:00
Julia Medina 77bd26a66d Non mutable default in Spider.custom_settings 2014-09-01 21:56:57 -03:00
Julia Medina 16e62e9c9b Per-spider settings documentation 2014-09-01 21:56:57 -03:00
Julia Medina 9ef3972cfb Per-spider settings tests 2014-09-01 21:56:57 -03:00
Julia Medina 4932ec43a7 Per-spider settings implementation 2014-09-01 21:56:57 -03:00
Daniel Graña ccde3317d7 Merge pull request #816 from Curita/api-cleanup 
GSoC API cleanup
 2014-09-01 21:55:36 -03:00
Pablo Hoffman 620dbe7116 Merge pull request #865 from yakxxx/master 
SgmlLinkExtractor - fix for parsing <area> tag with Unicode present
 2014-08-29 16:18:44 -03:00
Mikhail Korobov 8aa731c87f Merge pull request #866 from adamdonahue/patch-1 
Fix typo
 2014-08-29 06:59:47 +06:00
Adam Donahue d92914d297 Fix typo 2014-08-28 20:30:50 -04:00
yakxxx e4689556f0 SgmlLinkExtractor - fix for parsing <area> tag with Unicode present 2014-08-28 18:55:58 +02:00
Mikhail Korobov 774ab74ad2 Merge pull request #864 from younghz/master 
Duplicate comma in request-response.rst
 2014-08-28 18:52:51 +06:00
Uyounghz d49766a6ac Duplicate comma in request-response.rst 2014-08-28 19:58:58 +08:00
Mikhail Korobov c4f9e9d8e7 Merge pull request #856 from eliasdorneles/fix-embed-ipython-shell 
fix IPython shell scope issue by using IPython.embed()
 2014-08-20 00:16:33 +06:00
Pablo Hoffman 8360380db0 removed scrapy-ws.py, moved to scrapy-jsonrpc package 2014-08-19 10:30:25 -03:00
Elias Dorneles 6f50cf55a4 fix IPython shell scope issue and load IPython user config 2014-08-18 18:41:05 -03:00
Daniel Graña 841dd5f1f5 Update webservice.rst 2014-08-18 17:48:01 -03:00
Daniel Graña d684ecad7b Merge pull request #846 from rocioar/master 
fix dont_merge_cookies bad behaviour when set to false on meta
 2014-08-18 13:54:11 -03:00
Daniel Graña 94d00b2a26 Merge branch 'jsonrpc-split' 2014-08-15 23:28:26 -03:00
Daniel Graña a9292cfab7 jsonrpc webservice moved to https://github.com/scrapy/scrapy-jsonrpc repository 2014-08-15 23:28:13 -03:00
Rocio Aramberri 51b0bd281d fix dont settings on meta behaviour, add docs and tests 2014-08-15 13:47:42 -07:00
Julia Medina 6339864f95 Minor refactor in the docs and functions used in the shell command 2014-08-14 12:32:37 -03:00
Julia Medina 70f2010db1 Change error type when updating frozen settings 2014-08-14 11:59:25 -03:00
Julia Medina 3547ca6e61 Add example on running spiders outside projects 2014-08-14 11:50:33 -03:00
Julia Medina 419026615f Deprecate Crawler.spiders attribute 2014-08-14 09:19:41 -03:00
Julia Medina c90977ca98 Drop support for scrapy.project.crawler (And scrapy.stats consequently) 2014-08-12 14:02:56 -03:00
Julia Medina 9cbbfd8b04 Adjust spiders' utils to new SpiderManager API 2014-08-12 14:02:56 -03:00
Julia Medina 900a487682 Support multiple simultaneous LogObservers listening different crawlers 2014-08-12 14:02:56 -03:00
Julia Medina 89df18bd2b Fix usage of crawler_process in ScrapyCommands 2014-08-12 14:02:56 -03:00
Julia Medina d0edad4b0b Drop support for ScrapyCommand.crawler property 2014-08-12 14:02:56 -03:00
Julia Medina 870438e5f4 Update tests utils, fixing get_crawler and removing docrawl 2014-08-12 14:02:56 -03:00
Julia Medina d40273561d CrawlerProcess cleanup changes 2014-08-12 14:02:55 -03:00
Julia Medina 980e30a187 Crawler interface cleanup 2014-08-12 14:02:55 -03:00
Julia Medina d7038b2a13 SpiderManager interface cleanup 2014-08-12 14:02:55 -03:00
Julia Medina 39c6a80f9d Both getdict and getlist return copies of the requested values 2014-08-12 14:02:55 -03:00
Julia Medina 3ae971468f Add Settings.copy, freeze and frozencopy method 2014-08-12 14:02:55 -03:00
Julia Medina a995727117 Connect spider_closed signal after a crawler is bound to a Spider 2014-08-12 14:02:55 -03:00
Julia Medina eb0253e530 Update from_crawler method as well as set_crawler on CrawlSpider 2014-08-11 11:24:01 -03:00
Julia Medina 84fa004793 Add from_crawler class method to base Spider 2014-08-11 11:23:57 -03:00
Daniel Graña 8fece4b0b8 Add 0.24.4 release notes 2014-08-09 17:21:20 -03:00
Daniel Graña 02dd4a56a2 fix requirement typo 2014-08-09 00:44:48 -03:00
Daniel Graña 37787081d8 precise ships zope.interface 3.6.1 2014-08-09 00:37:08 -03:00
Daniel Graña 4badcc077d Add 0.24.3 release notes 2014-08-09 00:31:26 -03:00
Daniel Graña 0254f58763 Merge branch 'moderm-setuppy' 2014-08-08 18:09:43 -03:00
Daniel Graña 0772201ac8 Update installation docs 2014-08-08 18:07:38 -03:00
Daniel Graña 029c51acef There is a trove classifier for Scrapy framework! 
Added by https://bitbucket.org/pypa/pypi/issue/179
 2014-08-05 14:43:13 -03:00
Shane c2497e660b Merge pull request #841 from cyberplant/patch-1 
Fixed buildbot tests, after 242c085
 2014-08-05 12:31:20 +01:00