Web servers offer the concept of a default document, so when you navigate to a sub level of a site without including a document name, the correct document will be displayed. This means two different unique URIs can actually point to the same resource, and should be handled accordingly.
HTTP Crawler Test Site