RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,790 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

154 Commits

Author SHA1 Message Date
Daniel Graña db714f5a07
Merge pull request #3283 from CCInCharge/issue3247 
Fix #3247: Allow scrapy.FormRequest.from_response method to handle duplicate keys
 2018-08-02 14:49:14 -03:00
CCInCharge 782f866572 Fix #3247: Allow scrapy.FormRequest.from_response method to handle duplicate keys 2018-07-25 21:46:42 -07:00
Eugenio Lacuesta 6a38fc39f8 Include flags when copying requests 2018-07-19 11:56:23 -03:00
Pengyu Chen e75f721c04
Added: Allowing optional arguments for `scrapy.http.cookies.CookieJar.clear` 2018-04-23 22:08:28 +08:00
Viral Mehta dd064413a4 corrected syntax error in XPath 2018-03-19 19:28:41 +05:30
Viral Mehta a5acc9373f Resolving Comments 2018-03-19 18:19:39 +05:30
Viral Mehta e25e2afe17 Removed unnecessary print statements 2018-03-17 18:20:14 +05:30
Viral Mehta ff5f717f7a Fixed formatting issues 2018-03-17 18:17:48 +05:30
Viral Mehta d5b7ebcfdc Fixed bug FormRequest.from_response() clickdata ignores input[type=image] 2018-03-03 18:17:49 +05:30
NewUserHa acd2b8d43b [MRG+1] Fix part of issue #3128 - None should not be a valid type for 'url' in Response.follow (#3131) 
* fix one issue of issue#3128

because @kmike posted: 'If url is '', Scrapy should follow the same page, this is an intended behavior.'

*  fix one issue of issue#3128

because @kmike posted: 'If url is '', Scrapy should follow the same page, this is an intended behavior.'
 2018-02-22 03:37:26 +05:00
Daniel Graña 2371a2a0df Merge pull request #2789 from starrify/add-response-follow-tag-link 
[MRG+1] Added: Now supporting <link> tags in Response.follow
 2017-07-24 15:41:30 -03:00
Valdir Stumm Junior bb0bd691d9 Improve error message when callback is not callable 2017-07-24 11:12:09 -03:00
Pengyu CHEN b33e0d5a54 Added: Now supporting <link> tags in Response.follow 2017-06-14 17:09:42 +08:00
Valdir Stumm Junior 60727dedf6 verify if Request callback is callable 2017-05-31 15:00:38 -03:00
Harrison Gregg 26f723e4e6 Allow formdata value to be None to drop field generated from response 2017-05-15 12:25:27 -04:00
Omer Schleifer ff3e299eb0 [MRG+2] add flags to request (#2082) 
* add flags to request

* fxi test - add flags to request

* fix test(2) - add flags to request

* fix test(2) - add flags to request

* Updated test to reqser with flags field of request

* Updated documntation with flags field of request

* fix test identation

* fix test failed

* make the change backward comptaible

* remove  unrequired  spaces, fix documentation request flags

* remove  unrequired  space

* fx assert equal

* flags default is empty list

* Add flags to request

* add flags to request

* fxi test - add flags to request

* fix test(2) - add flags to request

* fix test(2) - add flags to request

* Updated test to reqser with flags field of request

* Updated documntation with flags field of request

* fix test identation

* fix test failed

* make the change backward comptaible

* remove  unrequired  spaces, fix documentation request flags

* remove  unrequired  space

* fx assert equal

* flags default is empty list

* add flags to request squashed commits
 2017-02-20 20:42:29 +06:00
Daniel Graña c68140e68a Merge pull request #2540 from scrapy/response-follow 
response.follow
 2017-02-20 11:21:21 -03:00
Mikhail Korobov 074caf434e FormRequest: handle whitespaces in action attribute properly 2017-02-16 02:13:39 +05:00
Mikhail Korobov 5b79c6a679 DOC document response.follow methods; expand the tutorial 2017-02-16 00:06:52 +05:00
Mikhail Korobov 2674f317df Response.follow 2017-02-15 04:39:47 +05:00
Mikhail Korobov 608c3f0c45 handle whitespace in response.follow; add tests 2017-02-15 04:17:41 +05:00
Mikhail Korobov 71dd5d0bf9 strip URL extracted from selectors (as per html5 standard) 2017-02-15 01:22:53 +05:00
Mikhail Korobov 877057fac0 initial response.follow implementation 2017-02-15 01:22:53 +05:00
Paul Tremberth 0cf6344cc2 Support kwargs for response.xpath() 2017-02-02 17:44:57 +01:00
Elias Dorneles 9c9690c76c add better messages for when response content isn't text (closes #2264) 2016-09-21 10:30:35 -03:00
Paul Tremberth d42a98d3b5 Use newer w3lib.url.safe_url_string() and re-enable HTTP request tests 2016-04-12 00:33:25 +02:00
Konstantin Lopuhin 617631f264 Fix method name: this always returns unicode keys and values 2016-02-18 10:10:16 +03:00
Konstantin Lopuhin 1b1092b7d0 add Headers.to_native_string_dict - useful when interfacing with other libraries 2016-02-15 19:59:47 +03:00
orangain 1f743996ff PY3: Implement some attributes of WrappedRequest required in Python 3 
This will fix #1770.
 2016-02-07 14:19:27 +09:00
Mikhail Korobov 3e080c3c52 call .text from .body_as_unicode() and not the other way around 2016-01-28 00:59:27 +05:00
Mikhail Korobov 4bcbb77bcc response.text. Fixes GH-1729. 2016-01-27 01:28:11 +05:00
Capi Etheriel 659715ecd9 implements FormRequest.from_response CSS support 2016-01-21 01:05:20 -02:00
Νικόλαος-Διγενής Καραγιάννης ebfdb9bb03 readable xpath with exslt 2015-11-14 23:24:07 +02:00
Νικόλαος-Διγενής Καραγιάννης 4f98be60be FormRequest: fix unicode xpath exception 2015-11-13 18:56:05 +02:00
Νικόλαος-Διγενής Καραγιάννης 2d25eab0df FormRequest: <input>'s default type must be text 2015-11-13 18:05:07 +02:00
Νικόλαος-Διγενής Καραγιάννης 650acad2b7 FormRequest: fix case-insensitive type attributes 2015-11-13 17:57:46 +02:00
Pengyu CHEN c34dbe955d fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormRequest.from_response` when there is a `<base>` tag) 2015-10-29 14:18:59 +08:00
Olaf Dietsche 937277e859 Make sure value attribute values are set. 2015-09-01 12:44:04 +02:00
Olaf Dietsche 121d7535be Allow button cotrols to submit form 2015-08-31 19:58:16 +02:00
Elias Dorneles 26ebccd37a upgrade parsel and use its function to instantiate root for finding form 2015-08-11 14:09:34 -03:00
Elias Dorneles 6287fc3109 remove lxmldocument dependency from http.request.form 2015-08-11 14:09:00 -03:00
Daniel Graña dba7e39f61 Do not break cookie parsing on non-utf8 headers 2015-08-03 10:53:40 -03:00
Daniel Graña 5f02ef82e8 PY3 port http cookies handling 2015-08-02 22:28:45 -03:00
Daniel Graña 786f62664b Merge pull request #1137 from DharmeshPandav/patch-1 
Update form.py to improve existing capability
 2015-07-31 11:00:57 -03:00
Mikhail Korobov e853d9e910 partial port of Request and Response 2015-07-25 13:08:44 +02:00
Mikhail Korobov 61cd27e5c7 Rename unicode_to_str and str_to_unicode method. Fixes GH-778. 2015-07-23 15:00:06 +02:00
mrpandav 1eccd34a70 adding feature improvement for selecting form using form-id, in addition to formname , formnumer before we go for xpath.. making it more idiomatic in nature 2015-04-12 11:11:28 +05:30
Dharmesh Pandav 5a96a16914 Update form.py to improve existing capability 
Add capability to search HTML Form using formid when using `FormRequest.from_response()`

refrenced issue :https://github.com/scrapy/scrapy/issues/1136
 2015-04-06 14:44:47 +05:30
nyov dc88be75e2 more python3 compatibility changes for urllib 2015-04-01 19:50:56 +00:00
nyov 84edc2ebc7 Add Response.urljoin() testcase 
and add evaluation of base-url for HtmlResponse.
 2015-03-27 05:59:01 +00:00