RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,497 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

962 Commits

Author SHA1 Message Date
Mikhail Korobov bdbca1e2d7 DOC request queue memory usage 2014-09-21 07:30:44 +06:00
Mikhail Korobov bc0f481a73 DOC bring back notes about multiple spiders per process because it is now documented how to do that 2014-09-21 07:12:01 +06:00
Mikhail Korobov a122fdbfea Update leaks.rst: there is now only a single spider in a process. 2014-09-21 06:54:00 +06:00
Mikhail Korobov e435b3e3a3 DOC simplify extension docs 2014-09-21 00:19:24 +06:00
John-Scott Atlakson a312ebfb43 Update request-response.rst 
Fixed minor typo
 2014-09-14 22:06:31 +06:00
Mikael Åhlén 47b6dff9f1 Allow to specify the quotechar in CSVFeedSpider 2014-09-13 02:14:57 +02:00
Julia Medina 16e62e9c9b Per-spider settings documentation 2014-09-01 21:56:57 -03:00
Daniel Graña ccde3317d7 Merge pull request #816 from Curita/api-cleanup 
GSoC API cleanup
 2014-09-01 21:55:36 -03:00
Mikhail Korobov 774ab74ad2 Merge pull request #864 from younghz/master 
Duplicate comma in request-response.rst
 2014-08-28 18:52:51 +06:00
Uyounghz d49766a6ac Duplicate comma in request-response.rst 2014-08-28 19:58:58 +08:00
Daniel Graña 841dd5f1f5 Update webservice.rst 2014-08-18 17:48:01 -03:00
Daniel Graña d684ecad7b Merge pull request #846 from rocioar/master 
fix dont_merge_cookies bad behaviour when set to false on meta
 2014-08-18 13:54:11 -03:00
Daniel Graña a9292cfab7 jsonrpc webservice moved to https://github.com/scrapy/scrapy-jsonrpc repository 2014-08-15 23:28:13 -03:00
Rocio Aramberri 51b0bd281d fix dont settings on meta behaviour, add docs and tests 2014-08-15 13:47:42 -07:00
Julia Medina 3547ca6e61 Add example on running spiders outside projects 2014-08-14 11:50:33 -03:00
Julia Medina 419026615f Deprecate Crawler.spiders attribute 2014-08-14 09:19:41 -03:00
Julia Medina c90977ca98 Drop support for scrapy.project.crawler (And scrapy.stats consequently) 2014-08-12 14:02:56 -03:00
Julia Medina 900a487682 Support multiple simultaneous LogObservers listening different crawlers 2014-08-12 14:02:56 -03:00
Julia Medina d40273561d CrawlerProcess cleanup changes 2014-08-12 14:02:55 -03:00
Julia Medina 980e30a187 Crawler interface cleanup 2014-08-12 14:02:55 -03:00
Julia Medina d7038b2a13 SpiderManager interface cleanup 2014-08-12 14:02:55 -03:00
Julia Medina 39c6a80f9d Both getdict and getlist return copies of the requested values 2014-08-12 14:02:55 -03:00
Julia Medina 3ae971468f Add Settings.copy, freeze and frozencopy method 2014-08-12 14:02:55 -03:00
Julia Medina 84fa004793 Add from_crawler class method to base Spider 2014-08-11 11:23:57 -03:00
Daniel Graña 8fece4b0b8 Add 0.24.4 release notes 2014-08-09 17:21:20 -03:00
Daniel Graña 4badcc077d Add 0.24.3 release notes 2014-08-09 00:31:26 -03:00
Daniel Graña 0772201ac8 Update installation docs 2014-08-08 18:07:38 -03:00
Daniel Graña 1a96cd5a3d update tox usage on contributing.rst 2014-07-31 12:28:35 -03:00
Daniel Graña 242c08553f move tests under root dir 2014-07-30 19:45:08 -03:00
Lisimia b6b44a33f7 Update shell.rst typo 
There was a typo. 
"start" was mistyped as "star"
 2014-07-28 12:26:13 -04:00
Elias Dorneles c298eaf164 removes weird indentation in the shell results 2014-07-23 20:42:36 -03:00
Elias Dorneles 514e8582d0 improved explanations, clarified blog post as source, added link for XPath string functions in the spec 2014-07-23 19:43:57 -03:00
Elias Dorneles 80de43ddab adding some xpath tips to selectors docs 2014-07-22 23:20:53 -03:00
Rolando Espinoza e441a01818 Updated input/ouput processor example according to #560. 2014-07-16 09:04:48 -04:00
Roy Crihfield 1a950e8f5a Fix typo in commands.rst 2014-07-10 14:48:15 -03:00
Daniel Graña 379f6ee6b2 Add 0.24.2 release notes 2014-07-08 05:53:35 -03:00
Zbigniew Siciarz 0466e8cb7a Fixed Python syntax in tutorial. 2014-07-04 10:38:01 +02:00
Mikhail Korobov 00cd4f0fa5 deprecate SgmlLinkExtractor 2014-07-02 23:15:17 +06:00
Mikhail Korobov 173a9dfe86 DOC fix formatting in release notes 2014-06-28 13:28:57 +06:00
Daniel Graña 284a2a06c9 Add 0.24.1 release notes 2014-06-27 12:03:37 -03:00
Daniel Graña 77c8284fa5 set 0.24.0 release date 2014-06-26 11:45:32 -03:00
Daniel Graña ee33efac39 Add 0.24.0 release notes 2014-06-26 10:29:47 -03:00
Daniel Graña 436c1c8980 address latest comments 2014-06-25 15:12:58 -03:00
Daniel Graña 2ad8db6ae6 Merge pull request #761 from dangra/lxmlextractor 
Promote LxmlLinkExtractor as LxmlExtractor
 2014-06-25 15:07:02 -03:00
Daniel Graña a9ecef5662 promote LxmlLinkExtractor as default in docs 2014-06-25 14:34:30 -03:00
Daniel Graña 5b2faf61c3 recognize jl extension as jsonlines exporter and update docs 2014-06-25 13:55:15 -03:00
Daniel Graña 1b32ece918 Merge branch 'settings-cleanup' of https://github.com/Curita/scrapy into Curita-settings-cleanup 2014-06-25 02:55:20 -03:00
Daniel Graña 499438ec44 indent parsed-literal as part of ordered list 2014-06-24 10:32:56 -03:00
Pablo Hoffman ee5c3209af doc: use |version| substitution in ubuntu packages 2014-06-24 09:52:51 -03:00
Paul Tremberth 90e69141d3 Add doc on LxmlLinkExtractor class 2014-06-23 22:35:56 +02:00
Daniel Graña 05ffca2781 Merge pull request #626 from dangra/500-leveldb-cache 
[WIP] Add a LevelDB cache backend
 2014-06-23 15:30:13 -03:00
Daniel Graña a90fd81c97 add leveldb cache backend docs 2014-06-23 14:37:29 -03:00
Pablo Hoffman 7fd8a1a387 minor change to request_scheduled signal doc 2014-06-14 20:51:16 -03:00
Nikolaos-Digenis Karagiannis 5f79940295 Document signal "request_scheduled" 2014-06-14 11:08:18 +03:00
Pablo Hoffman ad11fc01cb crate release notes for 0.24 and #699 to it 2014-06-10 18:46:53 -03:00
Julia Medina ed033854e2 setmodule helper method on Settings class 2014-06-10 11:26:50 -03:00
Julia Medina bdca06240c Fix settings repr on the logs of the shell and tutorial docs topics 2014-06-10 11:26:50 -03:00
Julia Medina 8a3a6236d9 Settings topic updated 2014-06-10 11:26:50 -03:00
Julia Medina 00e6470ed0 Fix and extend the documentation of the new Settings api 2014-06-10 11:26:38 -03:00
Daniel Graña 9830efb946 Merge pull request #697 from allait/master 
Add a note about reporting security issues
 2014-06-03 11:13:13 -03:00
Daniel Graña 7752074c51 Merge pull request #719 from Digenis/master 
document spider.closed() shortcut
 2014-06-03 10:01:15 -03:00
Daniel Graña 0467ad7719 Merge pull request #699 from allait/telnet-port 
Bind telnet console and webservice to 127.0.0.1 by default
 2014-06-03 09:51:16 -03:00
Daniel Graña 8a1c67fdaa Merge pull request #724 from rafallo/master 
item_dropped signal should pass response arg as item_scraped does
 2014-06-03 09:48:06 -03:00
Alexander Chekunkov fa5a67729b DOWNLOADER setting 2014-06-02 13:11:02 +03:00
Alexander Chekunkov 1fba64d34e DOWNLOADER setting 2014-06-02 13:05:22 +03:00
Rafal Jagoda ad6e449c63 Merge remote-tracking branch 'upstream/master' 2014-05-19 16:18:33 +01:00
Nikolaos-Digenis Karagiannis 24073a2624 document spider.closed() shortcut 2014-05-12 22:25:18 +03:00
Rendaw 8bdb6e2e3e Elaborated request priority value. 2014-05-07 19:14:45 +09:00
Carlos Rivera 946b854ddf grammatical issue 2014-05-06 15:41:59 -05:00
Rafal Jagoda 9d226e6266 add response arg to item_dropped signal handlers 
#710
 2014-05-02 14:42:01 +01:00
Alexander Chekunkov 4aa6179af7 added short RFPDupeFilter.request_fingerprint interface description 2014-04-26 15:46:53 +03:00
Alexander Chekunkov baaa077725 added note about RFPDupeFilter.request_fingerprint overriding to the settings documentation 2014-04-26 15:43:12 +03:00
Pablo Hoffman 3932877bcd signals doc: make argument order more consistent with code (although it doesn't matter in practice) 2014-04-25 16:03:32 -03:00
Pablo Hoffman eb07e09166 Merge pull request #663 from pawl/patch-1 
fixed typo
 2014-04-24 17:59:36 -04:00
Mikhail Korobov bdea071af3 DOC selectors.rst cleanup 2014-04-24 22:54:17 +06:00
ncp1113 5f18816428 for loops have to have a : at the end of the line 
changed 3 instances
 2014-04-24 22:50:21 +06:00
Daniel Graña 18412d75b5 Ammend example nesting selectors 2014-04-24 10:32:17 -03:00
Daniel Graña b4593c2ae7 document shortcuts in TextResponse class 2014-04-24 00:15:00 -03:00
Daniel Graña 1117687c47 update docs 2014-04-23 23:39:58 -03:00
Alexey Bezhan 89159779d0 Bind telnet console and webservice to 127.0.0.1 by default 2014-04-17 15:16:58 +01:00
Alexey Bezhan 6d6282da30 Add a note about reporting security issues 2014-04-17 14:41:21 +01:00
Mikhail Korobov 2d3803672b DOC use top-level shortcuts in docs 2014-04-15 01:09:35 +06:00
Julia Medina 80081054a2 Fix broken links in documentation 2014-04-09 18:57:52 -03:00
Julia Medina f0a0d0bc22 Ignore known broken links in docs linkcheck 2014-04-09 18:46:21 -03:00
Ana Sabina Uban 73109bf950 Fixed SgmlLinkExtractor constructor to properly handle both string and list parameters (attrs, tags, deny_extensions) 2014-03-27 16:12:00 +02:00
Paul Brown a1ee354609 fixed typo 2014-03-20 15:16:48 -05:00
tpeng fd7b40e892 add message when raise IngoreReques; fix item_scraped document 2014-03-18 15:23:25 +08:00
Julia Medina b9e2aad874 Doc for disabling download handler 2014-03-12 23:21:33 -03:00
Julia Medina ca1c1a82b5 FormRequest doc improvements 
Clickdata doc enhancements:
 * Fix xml attributes mention
 * nr is 0-indexed reference
 2014-03-12 12:34:50 -03:00
Julia Medina e29ab4d112 New doc: clickdata in Formrequest.from_response 
Documentation about:
 * clickdata parameter in Formrequest.from_response
 * nr attribute in clickdata dict
 * default behaviour when clickdata is None
 2014-03-12 06:43:50 -03:00
Nikolaos-Digenis Karagiannis 8ac9d40665 ItemLoader doc: missing args in replace_value() 2014-03-09 15:16:57 +02:00
Pablo Hoffman 0bc2cbaf09 Merge pull request #613 from breno/master 
Update scrapy command line doc with additional scrapy parse options
 2014-03-07 10:35:58 -08:00
Mikhail Korobov d8fb4a35db Merge pull request #627 from Digenis/master 
SpiderMW doc typo: SWP request, response
 2014-03-07 12:41:39 +05:00
Capi Etheriel 72b6c96d9a Running lucasdemarchi/codespell to fix typos in docs 2014-03-06 12:40:55 -03:00
Nikolaos-Digenis Karagiannis 4335420f40 SpiderMW doc typo: SWP request, response 2014-03-06 16:09:37 +02:00
Alexey Bezhan 210a0a6fe1 Fix some typos, whitespace and small errors in docs 2014-02-27 18:02:22 +00:00
Alexey Bezhan 2e59d77580 Clarify MapCompose documentation 2014-02-27 17:56:14 +00:00
Cash Costello c532e13dcd Added missing word in practices.rst 2014-02-27 13:08:36 -02:00
Pablo Hoffman 6d8b7b29ef remove unused setting: DOWNLOADER_DEBUG 2014-02-27 12:16:05 -02:00
Breno Colom 551c1c5eae Update scrapy command line doc with additional scrapy parse options 2014-02-24 18:10:51 -05:00