mirror of https://github.com/scrapy/scrapy.git
some notes about functional libraries
This commit is contained in:
parent
5436fd74a4
commit
0354bbc5e8
|
|
@ -93,3 +93,27 @@ Scrapy PRs
|
|||
https://github.com/scrapy/scrapy/pull/1012
|
||||
|
||||
https://github.com/scrapy/scrapy/pull/1016
|
||||
|
||||
Library notes
|
||||
-------------
|
||||
|
||||
fn.py is nice, but it looks very unusual.
|
||||
|
||||
toolz has a cytoolz counterpart which is implemented in Cython. I was using
|
||||
cytoolz just to speedup my Python code - with cytoolz code becomes both shorter
|
||||
and faster.
|
||||
|
||||
[cy]toolz.curried is nice; it allows to write e.g.
|
||||
|
||||
>>> from cytoolz.curried import compose, map, unique
|
||||
>>> compose(unique, map(unicode.strip))
|
||||
|
||||
making MapCompose unnecessary - function can be made working on a sequence
|
||||
by creating a curried map. Specializing map with a result of compose
|
||||
``map(compose(...))`` is also possible; it becomes the same as Scrapy
|
||||
MapCompose.
|
||||
|
||||
funcy provides helper functions useful for data extraction; they cover
|
||||
built-in Join and many of those proposed in
|
||||
https://github.com/scrapy/scrapy/pull/1012 like regex-based extraction.
|
||||
See e.g. http://funcy.readthedocs.org/en/stable/strings.html.
|
||||
|
|
|
|||
Loading…
Reference in New Issue