Information on Crawling

What is the Kyluka Crawler?

The Kyluka crawler is a robot that indexes publicly available information on the web. It's similar in concept to the Googlebot or Inktomi/Yahoo's Slurp.

To stop the crawler from crawling all or part of your site, create a robots.txt file as described in the Robot Exclusion Standard. Use the word kyluka to identify the crawler in a robots.txt file.

If you have questions please send them to crawl@kyluka.com.