RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,368 Commits 49 Branches 132 Tags 158 MiB
Commit Graph

3368 Commits

Author SHA1 Message Date
Daniel Graña c43931ea8c Use latest pyOpenSSL for all travis tests environments 
SSLv2 was removed from OpenSSL 1.0 and above but it is still referenced
by pyOpenSSL < 0.13. Travis workers are precise hosts with OpenSSL 1.0
and pyOpenSSL 0.12 (!) with a debian patch to workaround this problem
that is not present in pyOpenSSL 0.12 shipped by PyPi.

Trying to install pyOpenSSL 0.10 or 0.12 from packages at PyPi under a
system with OpenSSL >= 1.0 will success but fails at import time with a
message similar to:

    ImportError: .../lib/python2.7/site-packages/OpenSSL/SSL.so: undefined symbol: SSLv2_method
 2013-03-20 11:44:48 -03:00
Pablo Hoffman 9968f99e06 remove ssl from optional_features to simplify code, as it is now required. also deprecate optional_features set 2013-03-20 09:52:40 -03:00
Pablo Hoffman 320bdfe391 Merge pull request #269 from kalessin/settingdict 
added support for explicitly interpret a setting value as dict
 2013-03-19 11:10:54 -07:00
Martin Olveyra bf480015f4 added support generic python literals in settings, and for explicitly 
interpret a setting value as dict
 2013-03-19 16:02:53 -02:00
Pablo Hoffman d246b926bf Merge pull request #273 from plainas/master 
Accept ajax requests from other hosts (CORS support)
 2013-03-19 07:27:56 -07:00
Pedro a80ed769d9 allow remote ajax requests to the webservice 2013-03-19 12:00:49 +01:00
Pablo Hoffman b347c14b5f update engine status output on telnet console documentation 2013-03-18 19:12:12 -03:00
Pablo Hoffman 6f9f6f1f16 Merge pull request #271 from nramirezuy/scraper-6017 
Slots removed, now Scraper can handle just one spider
 2013-03-18 15:05:34 -07:00
Shane Evans 5c2a82f1f7 fix typo 2013-03-17 19:34:55 +00:00
Nicolás Ramírez 58975abeab Slots removed, now Scraper can handle just one spider 2013-03-15 11:11:51 -03:00
Pablo Hoffman e630126b82 scrapy deploy: return non-zero exit code if deploy fails 2013-03-14 16:47:35 -03:00
Pablo Hoffman bb20907254 minor updated to faq 2013-03-14 16:43:00 -03:00
Pablo Hoffman 098ccff862 added FAQ about error: "cannot import name crawler" 2013-03-14 12:57:59 -03:00
Pablo Hoffman a862f23376 added Nicolas Ramirez to AUTHORS 2013-03-14 12:44:39 -03:00
Pablo Hoffman 46b305ef7d Merge pull request #267 from nramirezuy/formrequest 
Override Request 'method' in FormRequest
 2013-03-14 08:43:53 -07:00
Nicolás Ramírez f043bba049 Override request method in FormRequest 
Changes proposed in the comments
 2013-03-14 10:39:17 -03:00
Pablo Hoffman a2e9d031f9 removed duplicated test 2013-03-14 10:32:19 -03:00
Pablo Hoffman 8391b36251 minor updates to contributing doc 2013-03-13 03:24:25 -03:00
Pablo Hoffman 51c301b3a2 added link to python binary libs, for windows installation 2013-03-13 03:18:33 -03:00
Pablo Hoffman 8e72730792 Merge pull request #261 from stav/allowed_domains 
allow spider allowed_domains to be set/tuple, #259
 2013-03-12 20:44:51 -07:00
Pablo Hoffman 296db1dc09 log (just once) when duplicate requests are filtered out. closes #105, #249 2013-03-12 19:28:43 -03:00
Pablo Hoffman a4507f62c3 sep-019: fixed typo 2013-03-12 18:44:09 -03:00
Steven Almeroth 650eda68da doc: add comment about commit history cleanliness 2013-03-10 18:51:04 -06:00
Steven Almeroth 5828179c5f remove over-testing and dict testing for test_utils_url.py 2013-03-10 18:30:49 -06:00
Steven Almeroth 1514b3b5db pylint clean-ups for test_utils_url.py 2013-03-10 16:47:24 -06:00
Steven Almeroth a613e15154 add tests for url_is_from_spider() with allowed_domains 2013-03-10 16:41:27 -06:00
Pablo Hoffman b2b256dc37 added nicolas to sep-019 authors, changed main title formatting for sep-019 and sep-020 2013-03-10 14:22:08 -03:00
Pablo Hoffman b11a1326b9 Merge pull request #264 from stav/sep 
add new sep-20 and update sep-19 with table at the top
 2013-03-08 11:23:15 -08:00
Steven Almeroth e66db6fa56 add new sep-20 and update sep-19 with table at the top 2013-03-08 13:04:11 -06:00
Pablo Hoffman eeb69d2f70 added #260 to release notes 2013-03-08 11:59:38 -02:00
Pablo Hoffman 42499c72b1 updated AUTHORS 2013-03-08 11:57:54 -02:00
Pablo Hoffman b5b37197bf Merge pull request #260 from llonchj/entry_point 
Commands module
 2013-03-08 05:49:08 -08:00
Pablo Hoffman b560f4d1c1 sep-019: remove TODO section (settled down with class method) 2013-03-08 09:57:12 -02:00
Pablo Hoffman a44757179b sep-019: other minor fixes 2013-03-07 19:38:12 -02:00
Pablo Hoffman c7add1e5bd sep-019: fix typos 2013-03-07 19:24:44 -02:00
Pablo Hoffman d806acdf50 sep-019: minor tide up 2013-03-07 12:39:40 -02:00
Pablo Hoffman 6f24b46115 added SEP-019 (per-spider settings) - first draft 2013-03-07 12:34:17 -02:00
Pablo Hoffman a027da0a2c Some changes to scrapy deploy command: 
- use branch names when constructing HG/GIT versions
- added -d flag for debugging (keeps build directory)
- improved command output (redirected setup.py stderr)
 2013-03-06 11:41:34 -02:00
Steven Almeroth b48ec1dce4 allow spider allowed_domains to be set/tuple, #259 2013-03-06 00:19:47 -06:00
Jordi Llonch 5b118ff4ab added documentation (experimental feature) 2013-03-06 06:36:23 +11:00
Jordi Llonch d9261f6c54 pluggable sub-commands for scrapy comand-line 2013-03-06 05:25:38 +11:00
Pablo Hoffman 19d0942c74 Merge branch 'shell' of git://github.com/stav/scrapy into stav-shell 2013-03-04 02:14:01 -02:00
Pablo Hoffman 3c8eef99cb docs/contributing: added note explaining what Scrapy contrib is 2013-03-04 01:35:17 -02:00
Pablo Hoffman 7dd360f39f Merge pull request #257 from stav/cleanups 
doc: fix typo in spider middleware
 2013-03-03 19:17:46 -08:00
Steven Almeroth 81111dd39a fetch command should catch IgnoreRequest exception 2013-03-02 20:13:14 -06:00
Steven Almeroth f62b6660d4 doc: fix typo in spider middleware 2013-03-02 19:46:31 -06:00
Pablo Hoffman d5d944fa44 Log overriden Scrapy settings when Scrapy starts. 
This is useful for debugging, to quickly find out which settings where
used for a specific spider run. dict settings are omitted for brevity.
DEBUG level is used for consistency with "Enabled
extensions/middlewares" lines (which share a similar purpose).
 2013-02-28 11:35:11 -02:00
Pablo Hoffman 8f3a509d44 remove debugging code 2013-02-27 03:52:55 -02:00
Pablo Hoffman 3a6b80259d arg_to_iter: replaced double call to isinstance with a single one. refs #248 2013-02-27 03:51:23 -02:00
Pablo Hoffman 2bbd92742b arg_to_iter: treat items the same way as dicts (ie. non iterables). fixes #248 2013-02-27 02:39:31 -02:00