有人认识这个爬虫吗 是干吗的?
18.118.49.184 – – [01/Dec/2021:23:53:25 +0800] “GET /robots.txt HTTP/2.0” 200 86 “-” “clark-crawler2/Nutch-1.19-SNAPSHOT” 18.118.49.184 – – [01/Dec/2021:23:53:26 +0800] “GET /page/1 HTTP/2.0” 200 7591 “-” “clark-crawler2/Nutch-1.19-SNAPSHOT”复制代码
看到之前也有一个MJJ问过这个问题:
https://hostloc.com/thread-898945-1-1.html