robots.txt editor downloads

RoboGen ROBOTS.TXT Editor

1.52 download

RoboGen is a visual editor for Robot Exclusion Files. It allows the user to quickly and easily create the ROBOTS.TXT files required to instruct web search engines which ...

WebSite Auditor Professional

4.58.6 download

... your site; -Find resources restricted from indexing (in robots.txt, by Noindex meta tag or Noindex X-Robots tag); ... landing pages in one place; -An XML Sitemap editor; -A robots.txt generator; -A reporting module; -A Domain ...