RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,439 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

2 Commits

Author SHA1 Message Date
Mikhail Korobov 36eec8f413 dont_obey_robotstxt meta key; don't process requests to /robots.txt 2014-09-23 00:10:43 +06:00
Daniel Graña 242c08553f move tests under root dir 2014-07-30 19:45:08 -03:00