The Discovery Problem

In The You On AI Field Guide

The existing discovery mechanisms fail for second-surplus artifacts for three converging reasons. First, volume: when a billion people can build software, the signal-to-noise ratio deteriorates past the point where ranking algorithms can separate valuable from trivial artifacts. Second, context-dependence: a tool valuable to one nurse in one clinic may be useless to a nurse in a different clinic with a different workflow, so popularity metrics poorly approximate value for any specific user. Third, failure-mode opacity: unlike a blog post, whose failure is typically visible to the reader, a software tool can appear to work while containing subtle defects that only testing or domain expertise reveals.

Adequate discovery mechanisms for the second surplus must therefore combine features that existing mechanisms do not combine. Categorization by problem rather than technology, because users think about problems they need solved, not about the stacks tools are built on. Recommendation informed by user context — profession, workflow, specific needs — rather than by popularity or behavioral similarity. Community curation by domain experts who can evaluate whether a tool serves its stated purpose, not by general algorithms whose signals are behavioral. And hybrid quality assurance that combines automated technical evaluation with human domain review, surfacing not just popular tools but tools that have passed both technical and contextual tests.

The discovery problem connects directly to the governance question: discovery mechanisms are not neutral, and the entities operating them exercise decisive influence over which second-surplus artifacts find their users. If discovery is controlled by the same platforms that provide the AI tools, the entire surplus is subject to capture at the discovery layer. If discovery is controlled by community-driven infrastructure, the surplus has a better chance of being deployed toward diverse user needs rather than channeled toward platform commercial interests. The design of discovery infrastructure is thus among the most consequential governance decisions the second surplus presents.

Origin

The problem is identified in this book as a structural feature of the second cognitive surplus, drawing on observations about the inadequacy of existing discovery mechanisms (app stores, search engines, social media) for the character and volume of AI-enabled creation. The underlying analysis of discovery in abundance draws on work by Yochai Benkler, Chris Anderson's long-tail framework, and research on recommendation systems.

Key Ideas

The volume threshold. At sufficient scale, ranking algorithms cannot distinguish signal from noise; new mechanisms are required.

Context-dependence of value. Second-surplus artifacts are often valuable to specific users in specific contexts; popularity is a poor proxy.

Failure-mode opacity. Software can appear functional while failing in ways that require domain expertise to detect.

The platform capture risk. Discovery infrastructure concentrates power; the entities controlling it shape which artifacts find users.

Hybrid discovery requirement. Adequate mechanisms combine automated technical assessment, community curation, and context-sensitive recommendation.