Web server not responding to shutdown request

Created by: walterl

After starting a scan on the OWASP BWA web-app and letting it run for almost 18 hours, Arachni has found about 150,000 pages. At this point the web server was still responding as expected. After clicking the "Shutdown" button, it seems that the web server has stopped responding to requests.

Any request of a valid URL seems to timeout without error (no 503 error, but not a byte of a response either). Even performing a raw HTTP "HEAD / HTTP/1.0\n\n" request returns nothing. Invalid URLs (e.g. http://arachni:4567/foo) immediately responds with a "Sinatra doesn’t know this ditty"-error.

There is no output in the terminal where the web server was started. The terminal where arachni_rpcd was started still prints a "INFO -- Call: dispatcher.stats [127.0.0.1]"-line every few seconds. One of the arachni_rpcd processes are also still using 90+% CPU.

Arachni was run from source, experimental branch at 996460d4.