crawler listening new request url
What is the current behavior? The crawler is closed when the queue is empty
What is the expected behavior? Is there a possibility to not to close crawler but to keep it listening new url to crawl?
Please tell us about your environment:
- Version: lastest
- Platform / OS version: Centos 7.1
- Node.js version: v8.4.0