Crawl stops on non-www URLs
What is the current behavior? If I specify a domain like eg. "http://www.domainname.com/" but the preferred domain settings on the server are without "www." then the crawling process stops.
The REVERSE is also valid unfortunately if I specify a DOMAIN that has "www" but I do not specify it eg. "http://domainname.com/" the crawling also STOPS.
If the current behavior is a bug, please provide the steps to reproduce
What is the expected behavior? Normally I expect it to recognize the domain name without "www"
What is the motivation / use case for changing the behavior?
Please tell us about your environment:
- Version: latest
- Platform / OS version: mac OS X
- Node.js version: latest