diff --git a/GSoC-2014-Draft-Ideas.rst b/GSoC-2014-Draft-Ideas.rst new file mode 100644 index 0000000..e51fccb --- /dev/null +++ b/GSoC-2014-Draft-Ideas.rst @@ -0,0 +1,40 @@ +Google Summer of Code - Draft Ideas +=================================== + +Better generator support +------------------------ + +TODO: + This one should be better specified. + +==================== =========== +Brief explanation Improve Scrapy API using generators +Expected results Scrapy should provide an easy way to build a single item from several pages, ... +Required skills Python, general understanding of async code, API design +Mentor(s) Mikhail Korobov, Rolando Espinoza +==================== =========== + +There are areas where Scrapy usability and efficiency can be improved by using generators, for example: + +* Integrate something like Rolando's https://github.com/darkrho/scrapy-inline-requests; +* ensure generators are not exhausted needlessly in various places; +* provide an easier alternative to spider_idle signal, something in line with https://github.com/scrapy/scrapy/issues/456 +* ... + +Reading list: + +* http://www.python.org/dev/peps/pep-0342/ +* http://www.tornadoweb.org/en/stable/gen.html +* http://twistedmatrix.com/documents/13.0.0/core/howto/defer.html +* https://twistedmatrix.com/trac/wiki/DeferredGenerator +Improve javascript integration +------------------------------ + +==================== =========== +Brief explanation Improve Javascript integration by using `Splash`_ to render and execute Javascript. +Expected results A Scrapy middleware to integrate with `Splash`_ +Required skills Scrapy +Mentor(s) Mikhail Korobov, Daniel Graña +==================== =========== + +.. _Splash: https://github.com/scrapinghub/splash diff --git a/GSoC-2014-Ideas.rst b/GSoC-2014-Ideas.rst index d350d82..0fe1cca 100644 --- a/GSoC-2014-Ideas.rst +++ b/GSoC-2014-Ideas.rst @@ -65,32 +65,6 @@ Mentor(s) Pablo Hoffman, Nicolas Ramirez .. _SEP-019: https://github.com/scrapy/scrapy/blob/master/sep/sep-019.rst -Better generator support ------------------------- - -TODO: - This one should be better specified. - -==================== =========== -Brief explanation Improve Scrapy API using generators -Expected results Scrapy should provide an easy way to build a single item from several pages, ... -Required skills Python, general understanding of async code, API design -Mentor(s) Mikhail Korobov, Rolando Espinoza -==================== =========== - -There are areas where Scrapy usability and efficiency can be improved by using generators, for example: - -* Integrate something like Rolando's https://github.com/darkrho/scrapy-inline-requests; -* ensure generators are not exhausted needlessly in various places; -* provide an easier alternative to spider_idle signal, something in line with https://github.com/scrapy/scrapy/issues/456 -* ... - -Reading list: - -* http://www.python.org/dev/peps/pep-0342/ -* http://www.tornadoweb.org/en/stable/gen.html -* http://twistedmatrix.com/documents/13.0.0/core/howto/defer.html -* https://twistedmatrix.com/trac/wiki/DeferredGenerator Python 3 support ---------------- diff --git a/GSoC-2014.md b/GSoC-2014.md index 602ec32..39bb11d 100644 --- a/GSoC-2014.md +++ b/GSoC-2014.md @@ -1,4 +1,5 @@ Scrapy is applying for [Google Summer of Code 2014](http://www.google-melange.com/gsoc/homepage/google/gsoc2014)! * [[GSoC 2014 Ideas]] -* [[GSoC 2014 Organization Application]] \ No newline at end of file +* [[GSoC 2014 Organization Application]] +* [[GSoC 2014 Draft Ideas]] diff --git a/GSoC-Draft-Ideas.rest b/GSoC-Draft-Ideas.rest deleted file mode 100644 index 4f964f0..0000000 --- a/GSoC-Draft-Ideas.rest +++ /dev/null @@ -1,14 +0,0 @@ -Google Summer of Code - Draft Ideas -=================================== - -Improve javascript integration ------------------------------- - -==================== =========== -Brief explanation Improve Javascript integration by using `Splash`_ to render and execute Javascript. -Expected results A Scrapy middleware to integrate with `Splash`_ -Required skills Scrapy -Mentor(s) Mikhail Korobov, Daniel Graña -==================== =========== - -.. _Splash: https://github.com/scrapinghub/splash \ No newline at end of file