There is a lot of discussion going on about ways for user agents (read browsers) can locate site metadata.  People are even arguing about what constitutes a site.  Beside the discussion within W3C TAG, RSS developers are discussing this topic with RSS feed discovery in mind.  Consensus seems to be moving away from using robots.txt style solution which uses fixed-URI.

Tim Berners-Lee wrote back in February:

The architecture of the web is that the space of identifiers on an http web site is owned by the owner of the domain name.  The owner, "publisher",  is free to allocate identifiers and define how they are served.

Any variation from this breaks the web.

Hogwash.

1. Web is just not that brittle.
2. Other solutions are not as easy.
3. User agents should protect themselves from unexpected data.
4. People will not revolt if W3C reserves some range of names if they are reasonably unique.

Simplest solution IMHO is to introduce a special file extension for metadata and a default file name for directory metadata.

For example, if ".w3c" file extension is used for metadata and default file name for directory metadata is empty string, metadata for the resource "/application/foobar.html" can be found in "/application/foobar.w3c" and metadata for the path "/application/" can be found in "/application/.w3c".

Add to this a hierarchical inheritance rule which basically say metadata not specific to a resource can be overriden by subpaths.  For convenience sake, subpaths starting with "_w3c" should be reserved.

Using this solution, my blog's RSS feed list can be located by fetching "https://blog.docuverse.com/.w3c".  Problem solved.

To me, current discussions are no different than discussions about where the toilet flush lever should be placed.  Should it be on the right-side because there are more right-handed people or at the center to be fair?  I say let the manufacturers place the damn lever anywhere convenient and noticeable.  'Users' will do the rest.