RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,884 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

4884 Commits

Author SHA1 Message Date
Paul Tremberth 5bf10cc8ff Fix tests on URL path encoding for links from latin1 document 
UTF-8 is to be used for path component.
Page encoding only affects query part of URLs.
 2016-04-07 22:43:47 +02:00
pawelmhm 8efd6d26ad [Backport][1.0] response_status_message should not fail on non-standard HTTP codes 
utility is used in retry middleware and it was failing to handle non-standard HTTP codes.
Instead of raising exceptions when passing through to_native_str it should return
"Unknown status" message.
 2016-04-06 20:21:26 +02:00
Paul Tremberth 38b09085eb Add support for Sphinx 1.4 
See http://www.sphinx-doc.org/en/stable/changes.html#release-1-4-released-mar-28-2016

sphinx_rtd_theme has become optional, needs to be added to reqs

https://github.com/sphinx-doc/sphinx/pull/2320 changes node entries tuples
to 5 values instead of 4

`sh` syntax highlighting added very locally in selectors.rst
because of this warning/error with Sphinx 1.4:

```
Warning, treated as error:
/home/paul/src/scrapy/docs/topics/selectors.rst:743:
WARNING: Could not lex literal_block as "python". Highlighting skipped.
```
 2016-03-30 17:55:20 +02:00
Paul Tremberth 26642388c3 Merge pull request #1875 from nanolab/patch-1 
[MRG+1] FS cache storage misuses mtime for expiration
 2016-03-30 10:47:27 +02:00
nanolab 5b98e9c4b5 Update httpcache.py 
It checks cache directory modification time, but have to check file modification time.
 2016-03-23 11:37:01 +02:00
Elias Dorneles 4b324a8f5b Merge pull request #1869 from eldios/patch-1 
Changed sel. to response. for clarity
 2016-03-18 10:38:12 -03:00
Lele 6d193e4dab Changed sel. to response. for clarity 
Changed sel. to response. to comply with the rest of the examples in the same section, to avoid confusion.
 2016-03-18 03:30:31 +01:00
Paul Tremberth a046ce843a Merge pull request #1753 from scrapy/release-1.0.5 
[MRG+1] Release 1.0.5
 2016-02-03 20:55:48 +01:00
Paul Tremberth fd0a0ce054 Add 1.0.5 release notes 2016-02-03 16:08:46 +01:00
Paul Tremberth c099220c3c Bump version: 1.0.4 → 1.0.5 2016-02-03 15:24:28 +01:00
Daniel Graña 0013080a3c Merge pull request #1690 from atmosx/buildbot-pytest 
[MRG+1] Changed buildbot makefile to use 'pytest'
 2016-01-19 13:32:49 -03:00
Elias Dorneles 9396e0ee3f Merge pull request #1669 from redapple/bogus-links 
[MRG+1] [1.0.x backport] [LinkExtractors] Ignore bogus links
 2016-01-19 13:48:00 -02:00
Panagiotis Atmatzidis 1f3d90ad46 Changed buildbot makefile to use 'pytest' 2016-01-19 09:03:01 +02:00
Mikhail Korobov 4b3ea07983 Merge pull request #1627 from Yedie-Fang/patch-1 
Update media-pipeline.rst
 2016-01-19 00:08:30 +05:00
Elias Dorneles 1987476f0c Merge pull request #1675 from kadimi/patch-1 
Update tutorial.rst
 2016-01-14 09:25:23 -02:00
Nabil Kadimi 808a9ea9fb Update tutorial.rst 2016-01-14 07:59:06 +00:00
Mikhail Korobov a4b103f0d2 Merge pull request #1643 from orangain/doc-downloader-middleware-base-1.0 
DOC: Add AjaxCrawlMiddleware to DOWNLOADER_MIDDLEWARES_BASE
 2016-01-13 12:57:02 +05:00
Andrew Scorpil 108195e780 [LinkExtractors] Ignore bogus links 
(rebased the code for scrapy 1.0 and made a few code improvements --nyov)
 2016-01-11 15:42:56 +01:00
Daniel Graña 1a0fbf5ee5 Yet another try to build 1.0.4 tag 2015-12-30 16:21:06 -03:00
Daniel Graña d758758ddc Another try to upload to pypi 2015-12-30 16:12:20 -03:00
Daniel Graña 93deea8d5a Bump version: 1.0.3 → 1.0.4 2015-12-30 15:48:49 -03:00
Daniel Graña bf5628199d Add 1.0.4 release notes 2015-12-30 15:48:49 -03:00
Daniel Graña 0797680bc4 Merge branch 'release-1.0.4' into 1.0 2015-12-30 15:40:19 -03:00
Alexander Sibiryakov 7dfa979197 Ignoring xlib/tx folder, depending on Twisted version. 2015-12-30 15:31:10 -03:00
Daniel Graña 6e42f0bf44 Run on new travis-ci infra 2015-12-30 15:23:11 -03:00
seales 823a1cc685 Spelling fixes 2015-12-30 15:23:11 -03:00
Νικόλαος-Διγενής Καραγιάννης da3c155b39 escape nodename in xmliter regex 2015-12-30 15:23:10 -03:00
Νικόλαος-Διγενής Καραγιάννης 4418fc3b86 test xml nodename with dots 2015-12-30 15:23:10 -03:00
Mikhail Korobov a55078cff2 TST don't use broken Pillow version in tests 2015-12-30 15:23:10 -03:00
Pablo Hoffman 86fc33060b disable log on version command. closes #1426 2015-12-30 15:23:10 -03:00
Pablo Hoffman db4c9fee4c disable log on startproject command 2015-12-30 15:23:10 -03:00
Daniel Graña df2b9448a9 Add PyPI download stats badge 2015-12-30 15:02:18 -03:00
Mikhail Korobov a83ab41ba0 don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch 2015-12-30 14:58:42 -03:00
Mikhail Korobov 73ac80d3ca Add Python 3 porting status badge to the README 2015-12-30 14:47:35 -03:00
Mikhail Korobov 97d080ee26 fixed RFPDupeFilter persistence 2015-12-30 14:47:08 -03:00
Mikhail Korobov 97f2fb302d TST a test to show that dupefilter persistence is not working 2015-12-30 14:47:01 -03:00
Daniel Graña d9b4850c44 explicit close file on file:// scheme handler 2015-12-30 14:46:52 -03:00
Νικόλαος-Διγενής Καραγιάννης c0d0734027 Disable dupefilter in shell 2015-12-30 14:42:41 -03:00
orangain aa239ade8a DOC: Add captions to toctrees which appear in sidebar 2015-12-30 14:35:26 -03:00
Rolando Espinoza 10eb400e25 DOC Removed pywin32 from install instructions as it's already declared as dependency. 2015-12-30 14:35:16 -03:00
Rolando Espinoza 1c3600abd7 Added installation notes about using Conda for Windows and other OSes. 2015-12-30 14:34:59 -03:00
David Chen 7f4ddd5d8d Fixed minor grammar issues. 2015-12-30 14:34:47 -03:00
Valdir Stumm Jr b71f6775c1 fixed a typo in the documentation. 2015-12-30 14:34:15 -03:00
Christian Pedersen 5456c0e610 Version 1 now exists 2015-12-30 14:34:06 -03:00
hoatle 0a1366ed8d fix another invalid xpath error 2015-12-30 14:33:19 -03:00
Hoat Le ca8d60f267 fix ValueError: Invalid XPath: //div/[id="not-exists"]/text() on selectors.rst 
>>> response.xpath('//div/[id="not-exists"]/text()').extract_first() is None
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/http/response/text.py", line 109, in xpath
    return self.selector.xpath(query)
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/selector/unified.py", line 100, in xpath
    raise ValueError(msg if six.PY3 else msg.encode("unicode_escape"))
ValueError: Invalid XPath: //div/[id="not-exists"]/text()
 2015-12-30 14:33:13 -03:00
smirecki 7067117a7b Typos corrections 
I've made a few small corrections, some spelling changes and typo fixes.
I've tried to respect regional spelling differences and avoided proposing hyphenating compound words.

 Please enter the commit message for your changes. Lines starting
 2015-12-30 14:32:54 -03:00
hy 32f115c4c6 fix typos in downloader-middleware.rst and exceptions.rst, middlware -> middleware 2015-12-30 14:30:59 -03:00
Julia Medina 23fda69f3a Add note to ubuntu install section about debian compatibility 2015-12-30 14:30:36 -03:00
Julia Medina 98b63eeba4 Replace alternative OSX install workaround with virtualenv 2015-12-30 14:30:28 -03:00