HTTP Headers

The response headers a site sends, and where it redirects to.

A full address works too; the domain is taken from it.

Many servers answer a crawler differently from a browser. This is what the request will claim to be, not proof of who is asking.