One thing about the discovery in the federated landscape is that it _helps_ to have a centralized place to find content and categorize it. Fortunately, we can solve this! It'd require a bit of implicit opt-in from content creators (basically anyone publishing content). In the IndieWeb and parts of the Fediverse, there's a concept of using WebSub. WebSub "provides a common mechanism for communication between *publishers* of any kind of Web content and their *subscribers*, based on HTTP webhooks" (from the specifications, emphasis mine). I use SuperFeedr currently and plan to add more "hubs" over time. These hubs, if used by a lot of people, can show the frequency and speed at which content is shared amongst our sites.
I'm considering making a reference implementation of a WebSub server that works with the expectations of the specification but also does a bit of crawling on the information that passes through it. I'll aim to have it crawl in the same regard that one could look at the content flying through it. I'm sure that there's some prior art around what constitutes as usual "discovery" information but things I'd want to look for are:
-
High reaction count (to show posts one might have not seen yet if it's in their interests)
- Age of posts (to help surface new posts)
- Interest categorization (tag surfacing, sentiment analysis, etc)
With this kind of information, one could subscribe to a feed emitted by this service about the "world" via this service. It'd require people to submit content from their site to this service, which is something I might be able to petition for outside of the IndieWeb by way of a "bridge". A service like this would have to operate pretty transparently and with the consent of the site owner. As mentioned, I'd want it to adhere to the robots rules when scanning this.
