tooldiscovery¶

Discovery layer providing tool registry, search strategies, and progressive documentation. This repository enables efficient tool discovery through multiple search approaches.

Packages¶

Motivation¶

• Keep discovery fast and cheap (token-efficient)
• Decouple search quality from core registry
• Support multiple search strategies (lexical, BM25, semantic)
• Provide progressive disclosure of tool documentation
• Offer a unified facade for most consumers (discovery)

discovery Package¶

The discovery package provides a simple API that composes index, search, semantic search, and documentation into one facade.

Example¶

import "github.com/jonwraymond/tooldiscovery/discovery"

disc, _ := discovery.New(discovery.Options{})
_ = disc.RegisterTool(tool, backend)

results, _ := disc.Search(context.Background(), "create issue", 5)
for _, r := range results {
  fmt.Println(r.ScoreType, r.Summary.ID)
}

registry Package¶

The registry package is a high-level helper for building MCP servers that combines index, search, local handlers, and MCP backend aggregation.

Example¶

import "github.com/jonwraymond/tooldiscovery/registry"

reg := registry.New(registry.Config{
  ServerInfo: registry.ServerInfo{Name: "my-mcp", Version: "1.0.0"},
})

_ = reg.RegisterLocalFunc(
  "echo",
  "Echo input",
  map[string]any{"type": "object"},
  func(ctx context.Context, args map[string]any) (any, error) { return args, nil },
)

_ = reg.Start(context.Background())
defer reg.Stop()

index Package¶

The index package provides the global registry and search layer for tools.

Core Responsibilities¶

• Register tools + backends
• Search by name/namespace/description/tags
• List namespaces
• Resolve tools by canonical ID

Example¶

import "github.com/jonwraymond/tooldiscovery/index"

idx := index.NewInMemoryIndex()

_ = idx.RegisterTool(tool, backend)

summaries, _ := idx.Search("repo", 5)
for _, s := range summaries {
  fmt.Println(s.ID, s.ShortDescription)
}

search Package¶

The search package provides BM25-based full-text search using Bleve.

Features¶

• BM25 ranking algorithm
• Field boosts (name: 4x, namespace: 2x, tags: 1x)
• Fuzzy matching support
• Pluggable into index via SearchStrategy interface

Example¶

import "github.com/jonwraymond/tooldiscovery/search"

searcher, _ := search.NewBM25Searcher(search.DefaultConfig())
defer searcher.Close()

// Index documents
searcher.Index(docs)

// Search
results, _ := searcher.Search("create issue", 10)

semantic Package¶

The semantic package provides embedding-based semantic search (optional).

Features¶

• Vector similarity search
• Configurable embedder interface
• Cosine similarity ranking
• Hybrid search support (combine with BM25)

tooldoc Package¶

The tooldoc package provides progressive documentation with multiple detail levels.

Detail Levels¶

Example¶

import "github.com/jonwraymond/tooldiscovery/tooldoc"

store := tooldoc.NewInMemoryStore()

// Get progressive documentation
doc, _ := store.GetDoc(toolID, tooldoc.DetailSchema)
fmt.Println(doc.Tool.InputSchema)

Schemas and Contracts¶

tooldiscovery defines data contracts for discovery payloads (summaries, documentation records, and results) and relies on toolfoundation for JSON Schema validation. See:

• schemas and contracts

Diagram¶

Search Strategy Layering¶

flowchart TB
    Query["Search Query"] --> Index["index.Search()"]
    Index --> Lexical["Lexical (default)"]
    Index --> BM25["search.BM25Searcher"]
    Index --> Semantic["semantic.Searcher"]

    Lexical --> Results
    BM25 --> Results
    Semantic --> Results

    Results["Ranked Results"]

Key Design Decisions¶

1. Pluggable strategies: Search implementations are swappable
2. Token efficiency: Summaries exclude schemas to reduce tokens
3. Progressive disclosure: Request only the detail level needed
4. Optional semantic: Vector search is opt-in (requires embeddings)

Links¶

• Repository
• Docs index
• Design notes
• User journey
• Schemas and contracts
• Architecture
• Concurrency
• Registry