The Web is not built around advance informed consent; there’s no agreement to terms before downloading a public file (besides basic protocol negotiations). This is one reason why “by using this site, you agree to our cookies, privacy policy, kidney harvesting, etc” notices won’t fly under the GDPR.
A website admin can’t set terms for downloading a linked document; the user-agent just makes a request and the server works with that data to deny or accept it. There’s no obligation for the UA to be honest or accurate.
Ultimately, nobody is forcing you to run a Web server; however, plenty of people have to use the Web. Respect for the UA is part of the agreement you make when joining a UA-centric network.
Should you disagree with the precedent set by the HTML Living Standard, nearly every Web Accessibility Initiative standard (users must be able to override and replace stylesheets, colors, distracting elements), the exceptions to e.g. the Content Security Policy in Webappsec standards to allow UA-initiated script injection, etc.: you’re always free to build your own alternative to the Web with your own server-centric standards.