HTTP Crawler Test Site
Status Codes
Features
CSS
HTML
JavaScript
Features
This page indexes HTTP features which have relevance to the crawling of a website.
Authentication
CDN / Alternative Hosts
Certificates
Cookies
Default Document
Downloads
HTTP Compression
Image Formats
Meta Refresh
Mime types and meta data
Paths
Query Strings
Redirect Chain
Redirect Loop
Referer Header
Request Headers
Spaces in URLs
Special Characters
Subdomains
URL Normalization
User Agent
HTTP Crawler Test Site