Robots.txt checker
Share
Similar tools
Check the status code returned by a website for a generated missing page.
Checks if a webpage is set to noindex, preventing it from appearing in search engines.
Get all HTTP headers returned by a URL. Analyze headers such as Server, Set-Cookie, Cache-Control, Content-Type, and more.
Popular tools
Identify doWhois Lookupmains or hosts associated with a specific IP address. Get detailed information about the host connected to any given IP.
Find A, AAAA, CNAME, MX, NS, TXT, SOA DNS records of a host.
Get detailed information about any IP address, including geolocation, ISP details, and more.
Check whether a website is using the new HTTP/2 protocol or not. Discover why migrating to HTTPS is essential to take full advantage of HTTP/2.