You could define a spider attribute that references a callback method
but if this method has a different name than your spider attribute,
the request serializer is not able to find it on the spider class.
With this commit we're fixing this behavior as we're searching for
callback references in the spider object itself instead of looking
for attributes with the same function's name, that could be different.
* add flags to request
* fxi test - add flags to request
* fix test(2) - add flags to request
* fix test(2) - add flags to request
* Updated test to reqser with flags field of request
* Updated documntation with flags field of request
* fix test identation
* fix test failed
* make the change backward comptaible
* remove unrequired spaces, fix documentation request flags
* remove unrequired space
* fx assert equal
* flags default is empty list
* Add flags to request
* add flags to request
* fxi test - add flags to request
* fix test(2) - add flags to request
* fix test(2) - add flags to request
* Updated test to reqser with flags field of request
* Updated documntation with flags field of request
* fix test identation
* fix test failed
* make the change backward comptaible
* remove unrequired spaces, fix documentation request flags
* remove unrequired space
* fx assert equal
* flags default is empty list
* add flags to request squashed commits
crawls.
* requests are serialized (using marshal by default) and stored on disk, using
one queue per priority
* request priorities must be integers now
* breadh-first and depth-first crawling orders can now be configured
through a new DEPTH_PRIORITY setting (see doc). backwards compatilibty with
SCHEDULER_ORDER was kept.
* requests that can't be serialized (for example, non serializable callbacks)
are always kept in memory queues
* adapted crawl spider to work with persitent scheduler
Jalil SA
Emmanuel Rondan
Eugenio Lacuesta
Ammar Najjar
Iván de Prado
Victor Torres
Eugenio Lacuesta
Andrey Rakhmatullin
Andrew Baxter
Omer Schleifer
Mikhail Korobov
Pablo Hoffman