+- Fazer Owners Club - Unofficial (https://foc-u.co.uk/mybb)
+-- Forum: General (https://foc-u.co.uk/mybb/forumdisplay.php?fid=65)
+--- Forum: General (https://foc-u.co.uk/mybb/forumdisplay.php?fid=69)
+--- Thread: Unknown Robot (/showthread.php?tid=68977)
Unknown Robot - slimwilly - 31-05-14
Anyone else noticed that at the foot of the home page ,,it shows logged on users,,
well there always seems to be an "unknown robot" logged on :eek
wonder if he rides a Fazer
Re: Unknown Robot - Tmation - 31-05-14
There is 10 of the buggers logged on now
Re: Unknown Robot - Buzz - 31-05-14
These are web crawlers, there are others in the user list like Baidu (a massive Chinese search engine) that will be eating up the site's bandwidth. Depending on the server access the Admins should be able to block these. More info here.
Re: Unknown Robot - mickvp - 02-06-14
(31-05-14, 04:43 PM)Buzz link Wrote: These are web crawlers, there are others in the user list like Baidu (a massive Chinese search engine) that will be eating up the site's bandwidth. Depending on the server access the Admins should be able to block these. More info here.
yes, you can of course block them in your robots.txt file (i assume thats what your article is about), but you need to be selective about what you block or you will muller your search rankings in google et al if you block too much. bit of a balancing act.