It turns out this robot was the scanner for the Internet Archive. Over the years I found the Internet Archive to be quite useful, in particular the waybackmachine. Whilst the hits I saw were links to external sites and fairly pointless, I'm happy to accept the hits as I'm sure the Internet Archive helps many, many people.
Kelvin Eldridge