RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,260 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

82 Commits

Author SHA1 Message Date
elacuesta 5dbeece8da [MRG+1] Drop py34 support - Update CI envs (#3892) 
* Drop py34 support

* Travis experiments

* More Travis experiments

* Bump Twisted version for py35+ (stretch)

* Remove Debian build

* Remove pinned lxml for Py34

* Fix merge error

* Remove unused tox env

* Add environment with pinned versions for py36

* Bump minimum Twisted version in py27; Envs with pinned versions for py27 and py35

* Add botocore as extra dep for py27 tests

* Update requirements-py2.txt

* Add botocore and Pillow as extra dependencies
 2019-08-07 12:36:52 +05:00
Renne Rocha a25e09ecdd Added constrain on lxml version based on Python version 2019-07-30 23:24:41 -03:00
Adrián Chaves 0da972339b Require Twisted<=19.2.0 for Python 3.4 2019-06-11 14:11:38 +02:00
Mikhail Korobov cdd04dfb1d declare Python 3.7 support in setup.py 2018-12-26 13:13:49 +05:00
Mikhail Korobov 25ac4691b4 require parsel 1.5+ 2018-09-12 17:57:27 +05:00
Eugenio Lacuesta 0d87e77afe Bump parsel dependency 2018-02-08 15:03:57 -03:00
Mikhail Korobov 1058169f0e setup.py: mention that we support PyPy. See GH-2213. 2017-12-25 15:31:07 +05:00
Hugo 44623687ab Drop support for EOL Python 3.3 2017-12-19 17:59:05 +02:00
Aditya 9d9d83a8c3 Use https links wherever possible 2017-10-28 16:24:40 +05:30
Konstantin Lopuhin b0a9236357 Use environment markers for custom PyPy requirements 2017-06-19 19:16:50 +03:00
Mikhail Korobov e1ceaf3b5f require w3lib 1.17+ 2017-02-15 00:32:44 +05:00
Paul Tremberth 1295c17a26 Bump parsel requirement to at least parsel v1.1 2017-02-02 17:47:22 +01:00
Paul Tremberth b3406677b9 Update classifiers in setup.py 2017-01-09 14:40:02 +01:00
Paul Tremberth 78d34cc44c Merge pull request #2454 from crasker/fix/low_parsel_version_number 
[MRG+1] Fix/low parsel version number
 2016-12-19 17:11:17 +01:00
zhouyc 2b3abdb700 update parsel version which will cause attribute error 2016-12-19 16:42:31 +08:00
nyov c8cf1a303d Bump Twisted dependency to 13.1.0 (released June 2013) 2016-12-01 22:02:11 +00:00
Ashish Kulkarni bb3b806467 Use w3lib.url.canonicalize_url() from w3lib 1.15.0 
Also remove code/imports which are now unused due to this change.

fixes #2157
 2016-08-16 17:42:16 +05:30
Paul Tremberth c8bff23751 Bump w3lib version dependency in setup.py 2016-04-29 10:29:37 +02:00
nyov 3787fec460 Remove duplicate code now handled by newer w3lib 
see f3029a6a10
 2016-03-30 14:58:03 +00:00
Mikhail Korobov 17d3bec699 declare Python 3 support in setup.py 2016-03-01 16:34:13 +05:00
Elias Dorneles 26ebccd37a upgrade parsel and use its function to instantiate root for finding form 2015-08-11 14:09:34 -03:00
Elias Dorneles 2fe6d128f5 upgrade parsel and using promoted root attribute 2015-08-11 14:09:16 -03:00
Elias Dorneles 17d7347a36 update minimal parsel version, add deprecated classes for csstranslator modules, fix test 2015-08-11 14:09:00 -03:00
Elias Dorneles ce21884a97 migrating scrapy Selector to use Parsel 2015-08-11 14:08:43 -03:00
Mikhail Korobov 75d3d03382 add service_identity to scrapy install_requires 2015-08-11 20:14:22 +05:00
Daniel Graña 776064aee1 PY3 port signals 2015-07-25 18:04:55 +02:00
Elias Dorneles 3d3633f3d2 favoring web scraping over screen scraping in the descriptions 2015-04-25 11:20:20 -03:00
Pablo Hoffman 5846d6154c emphasize web crawling over screen scraping on scrapy description. closes #586 2015-03-18 00:24:16 -03:00
Daniel Graña 0254f58763 Merge branch 'moderm-setuppy' 2014-08-08 18:09:43 -03:00
Daniel Graña 029c51acef There is a trove classifier for Scrapy framework! 
Added by https://bitbucket.org/pypa/pypi/issue/179
 2014-08-05 14:43:13 -03:00
Paul Tremberth 480cfa1994 Update w3lib requirement to 1.8.0 2014-08-04 16:01:28 +02:00
Daniel Graña fcd34b6561 set zip_safe=False 2014-08-01 16:26:08 -03:00
Daniel Graña 1fc4e59cf4 do not ship tests package 2014-08-01 16:23:37 -03:00
Daniel Graña a8f45dc6dd Modernize setup.py 2014-08-01 15:43:33 -03:00
Daniel Graña bfe02b019e Revert "limit Twisted support to pre-14.0.0 while #718 is fixed" 
This reverts commit 65f69e1609.
 2014-06-25 04:22:18 -03:00
Daniel Graña 65f69e1609 limit Twisted support to pre-14.0.0 while #718 is fixed 2014-06-02 13:27:20 -03:00
alexanderlukanin13 ab6496647c Added "six>=1.5.2" to requirements 2014-01-15 13:05:00 +06:00
Daniel Graña cfe588103c include_package_data is required to build wheels from published sources 2013-11-28 16:25:57 -02:00
Daniel Graña 2df8156431 Drop Python 2.6 support 2013-10-29 13:44:00 -02:00
alexanderlukanin13 192c323a0c wrong variable name 2013-10-18 11:30:48 +06:00
Daniel Graña 757e262267 update requirement for cssselect >= 0.9 2013-10-17 09:33:15 -02:00
Capi Etheriel bc17e9d412 Adds HtmlCSSSelector and XmlCSSSelector classes, cssselect as optional dependency. 
Ported .get() from _Element and .text_content() from HTMLMixin

Add CSS selectors to scrapy shell

Documenting CSS Selectors: Constructing selectors

Documenting CSS Selectors: Using Selectors

Make CSS Selectors a default feature.

Adds XPath powers to CSS Selectors and some syntactic sugar.

Removes methods copied over from lxml.html.HtmlMixin.

Updating docs to use new CSS Selector super powers.

Documenting CSS Selectors: Regular Expressions

Moving section after Nesting section, since it mentions it.

Documenting CSS Selectors: Nesting Selectors

Fix XPath specificity in lxml.selector.CSSSelectorMixin.text

Cleaning up unused stuff from cssel.py

Changing the behavior of lxml.selector.CSSSelectorMixin.text.

Concatenating all of the descendant text nodes is more useful
than returning it in pieces (there's xpath() if you need that).

Documenting CSS Selectors: CSS Selector objects

Documenting CSS Selectors: CSSSelectorList objects

Documenting CSS Selectors: HtmlCSSSelector objects

Documenting CSS Selectors: XmlCSSSelector objects

Fixing some documentations typos and errors

Enforcing the 80-char width lines

Tidying up CSS selectors and CSSSelectorMixin objects

Adding some missing references in documentation.

Fixing lxml.selector.CSSSelectorList.text
 2013-10-10 18:23:15 -02:00
Pablo Hoffman a9c3519897 updated required twisted version to 10.0 2013-10-01 14:07:38 -03:00
Pablo Hoffman d02da2f31f ported code to use queuelib 2013-04-23 17:48:09 -03:00
Daniel Graña 910effd145 get scrapy version from package data 2013-02-06 11:44:26 -02:00
Daniel Graña 5db45b3825 remove scrapyd, it was migrated to its own repository 2013-02-06 05:24:07 +00:00
Pablo Hoffman b9a96147ed setup.py: use README.rst to populate long_description 2012-11-25 22:22:33 -02:00
Daniel Graña d74a067227 require w3lib 1.2 or greater 2012-05-15 17:11:51 -03:00
Pablo Hoffman 4f28ffcb2c removed no longer needed dependency on simplejson 2012-04-10 16:01:36 -03:00
Pablo Hoffman 6e8edbd72e switched default selectors backend to lxml 2012-04-10 15:52:14 -03:00