The Domain Name System (DNS) has a tree structure or hierarchy, with each non-RR (resource record) node on the tree being a domain name. A subdomain is a domain that is part of a larger domain; the only domain that is not also a subdomain is the root domain.
For example, demo.cyotek.com is a subdomain of
cyotek.com,
sub.demo.cyotek.com is a subdomain of
demo.cyotek.com. However while demo2.cyotek.com
is a subdomain of cyotek.com it is not a subdomain of
demo.cyotek.com.
WebCopy includes support for automatically including sub or sibling domains as required. It also has a shortcut "Copy All Resources" option which allows linked resources (e.g. non-HTML) to be downloaded regardless of location.
HTTP Crawler Test Site