scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

crawler crawling framework hacktoberfest python scraping web-scraping web-scraping-python

Go to file

Daniel Grana 8fdf06ed08 Avoid sending Content-Length header when body is an empty string. Some sites can't handle "Content-Length: 0" header, but twisted HTTPClientFactory adds Content-Length header unless body is None. scrapy enforces request.body usage as string, using None is not possible. thanks Matt for report, See: http://groups.google.com/group/scrapy-users/browse_thread/thread/380ffa111879989e?hl=en --HG-- extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40841	2009-02-10 13:23:38 +00:00
scrapy	Avoid sending Content-Length header when body is an empty string.	2009-02-10 13:23:38 +00:00
sites	disabled docs app (redirecting to doc.scrapy.org instead)	2009-01-04 23:28:50 +00:00
.hgignore	dummy file to shift a revision	2009-05-04 18:16:20 +00:00

Daniel Grana 8fdf06ed08 Avoid sending Content-Length header when body is an empty string.

Some sites can't handle "Content-Length: 0" header, but twisted
HTTPClientFactory adds Content-Length header unless body is None.

scrapy enforces request.body usage as string, using None is not
possible.

thanks Matt for report, See:
http://groups.google.com/group/scrapy-users/browse_thread/thread/380ffa111879989e?hl=en

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40841

2009-02-10 13:23:38 +00:00

scrapy

Avoid sending Content-Length header when body is an empty string.

2009-02-10 13:23:38 +00:00

sites

disabled docs app (redirecting to doc.scrapy.org instead)

2009-01-04 23:28:50 +00:00

.hgignore

dummy file to shift a revision

2009-05-04 18:16:20 +00:00