Repo Tagger

Repo Tagger is an automated CLI tool that uses local Large Language Models (via Ollama) to analyze repositories within a Forgejo (or Gitea) organization and automatically classify them by assigning relevant topics/labels.

It uses an autonomous agent loop with tool-calling capabilities (list_directory and read_file) to explore the repository's codebase, read READMEs, security advisories, or exploit scripts, and answer a customizable set of taxonomy questions (e.g., related CVEs, targeted OS, software products, protocols, etc.).

Features

• AI-Powered Codebase Analysis: Leverages rig-core and Ollama to autonomously explore repository contents and determine appropriate tags.
• Smart State Tracking: Uses a local SQLite database to track processed commits. It automatically skips repositories that haven't changed since their last analysis. A configurable cooldown period prevents re-analysis of recently processed repositories, even if new commits were pushed.
• Label Registry with Proposals: A versioned taxonomy system where LLM-suggested labels can be auto-confirmed, queued for review, or run in dry-run mode.
• Interactive Label Management: Review, edit, accept, or reject new label proposals through a built-in terminal UI.
• Taxonomy Customization: Define your own categories, questions, and baseline examples for the LLM to use when tagging.
• Dry-Run & Safety Controls: Review all proposed modifications before pushing anything to the server. Binary content detection and path traversal protection built into the file-reader tool.
• Automatic Retry: Failed LLM responses are retried up to 10 times, with invalid JSON or schema violations logged to disk for debugging.
• Benchmarking: Built-in criterion benchmarks for performance-sensitive components.

Prerequisites

• Rust: Edition 2024 (latest stable toolchain recommended).
• LLM endpoint: Any OpenAI-compatible API (a local Ollama instance by default, http://localhost:11434).
• LLM Model: A model on the LLM endpoint that supports tool calling. The default is qwen3.6:35b. Note this tag is not published to the public Ollama library — ensure it is built/pulled locally before first run, or override it with llm_model.
• Forgejo / Gitea: A target instance, organization, and an API token with permissions to read repositories and write topics.

Configuration

Configuration is handled via environment variables. You can provide these by creating a .env file in your working directory.

# Required
forgejo_token=your_api_token_here
forgejo_url=https://git.yourdomain.com
forgejo_organization=your_target_org

# Optional (Defaults shown)
llm_base_url=http://localhost:11434/v1
llm_model=qwen3.6:35b
temperature=0.0
storage_path=./data
max_tokens=65536
max_tool_calls=30
max_file_bytes=32768

# Daemon configuration (optional)
# daemon_interval_minutes=30
# daemon_webhook_key=your-webhook-secret-key
# daemon_webhook_port=3000
# daemon_webhook_host=0.0.0.0
# daemon_webhook_callback_retry_hours=24
# daemon_callback_max_retries=10
# daemon_callback_retry_base_delay_secs=2

# Process defaults (overridable via CLI flags)
# apply_default=KeepLocal
# print_thinking_default=Quiet
# include_schema_default=Include
# progress_display_default=Bar  # or Quiet for CI-friendly text output
# min_analysis_interval=24h     # Minimum time between re-analyses (e.g. 2h, 90m, 3600s; plain number = hours; 0 to disable)

CLI Usage

Usage: repo-tagger <COMMAND>

Commands:
  print        Print the registry as prompt
  init         Initialize the registry
  list         List organization repositories, metadata and labeling status
  process      Initiate analysis pipeline on a single chosen repository
  process-all  Process all organization repositories in sequential order
  apply        Transmit locally saved analysis labels directly to the server repository
  apply-all    Transmit processed labels for all eligible repositories to Forgejo
  delete       Remove all assigned metadata topics from a single repository
  delete-all   Reset all organization repositories to contain empty topics
  labels       Label Registry taxonomy management
  daemon       Run as daemon: continuously process repositories and listen for webhooks
  help         Print this message or the help of the given subcommand(s)

Options:
  -h, --help
          Print help

Commands

Command	Description
init	Create the initial labels.json label registry in the storage directory.
list	Fetch all repos from the Forgejo org and display their local state (commit, status, active labels).
process <repo>	Initiate analysis pipeline on a single chosen repository.
process-all	Iterate over all org repos, analyzing each. Skips repos that haven't changed (smart tracking) or were analyzed recently (cooldown).
apply [repo]	Transmit locally saved analysis labels directly to the server repository.
apply-all	Push labels for every repo in Processed state.
delete <repo>	Remove all topics from a repository on Forgejo.
delete-all	Remove all topics from every org repository.
labels list	Expose all cataloged baseline and proposed taxonomies.
labels review	Enter interactive terminal evaluation for pending labels.
labels check	Check for label conflicts (differing only by dashes) in the database.
daemon	Run as a continuous service with periodic repository processing and webhook server support (POST /webhook/:key, GET /health).
print	Render the current label registry as a system prompt (for debugging).

Process / Process-All flags

Both process and process-all share these flags:

      --apply
          Submit the generated labels immediately to the server
      --no-apply
          Just store the generated labels locally
      --print-thinking
          Print the debugging tokens emitted by the model
      --no-print-thinking
          Don't print the debugging tokens emitted by the model
      --temperature <TEMPERATURE>
          Temperature for the model
      --include-schema
          Include the JSON schema in the LLM prompt
      --no-include-schema
          Don't include the JSON schema in the LLM prompt
      --apply-mode <APPLY_MODE>
          Show planned modifications without executing them [default: auto-submit] [possible values: dry-run, confirm, auto-submit]
      --auto-accept-labels <AUTO_ACCEPT_LABELS>
          How should new labels be handled [default: queue-proposal] [possible values: auto-confirm, auto-accept, queue-proposal, dry-run]
      --ignore-existing-reports
          Disable the usage of existing reports to force a full re-analysis
      --min-analysis-interval <MIN_ANALYSIS_INTERVAL>
          Minimum time since last analysis (e.g. 24h, 90m, 3600s, 1h30m; plain number = hours; 0 to disable)
      --apply-selection <APPLY_SELECTION>
          Analyse all repositories or only those with updates [default: only-updated] [possible values: all, only-updated]
      --max-callback-retries <MAX_CALLBACK_RETRIES>
          Max callback retry attempts [default: 10]
      --callback-retry-ttl-hours <CALLBACK_RETRY_TTL_HOURS>
          Callback retry TTL in hours [default: 24]
      --callback-retry-base-delay-secs <CALLBACK_RETRY_BASE_DELAY_SECS>
          Callback retry base delay in seconds [default: 2]

Defaults for several of these flags can be set via environment variables (see Configuration); CLI flags always take precedence. --progress-display is only available on process-all.

process-only flags:

      --output-file <OUTPUT_FILE>
          Export output directly to a designated JSON file

process-all-only flags:

      --progress-display <PROGRESS_DISPLAY>
          Progress display mode [default: bar] [possible values: bar, simple, quiet]
          bar:    Show progress bar with elapsed time and position
          simple: Show simple text progress suitable for CI/CD logs
          quiet:  Suppress all progress output
      --output-dir <OUTPUT_DIR>
          Export outputs directly to a designated directory

Daemon & Webhook

The daemon command runs repo-tagger as a long-lived service. It combines two modes of operation:

1. Periodic processing — every daemon_interval_minutes (default 60) it scans the organisation for repositories with new commits (or none seen before) and enqueues them for analysis, respecting the same cooldown rules as process-all.
2. Webhook-triggered processing — when a webhook key is configured, it runs an HTTP server that accepts webhook notifications (e.g. from Forgejo) and enqueues the referenced repository for immediate analysis. Without a key the daemon starts without the HTTP server and only performs periodic processing.

Both paths share a single in-memory queue with deduplication, so the same repository is only analysed once at a time even if webhook and periodic scanner fire simultaneously.

Configuration

Variable	CLI flag	Type	Default	Description
daemon_webhook_key	--webhook-key	string	none (server disabled)	Shared secret that must appear in the webhook URL path (/webhook/<key>). Requests with the wrong key are rejected with 401. When unset, the daemon runs without an HTTP webhook server.
daemon_webhook_host	--webhook-host	string	0.0.0.0	Bind address for the webhook server.
daemon_webhook_port	--webhook-port	int	3000	TCP port the webhook server listens on.
daemon_interval_minutes	--interval-minutes	int	60	Interval between periodic organisation-wide scans.
daemon_callback_max_retries	--max-callback-retries	int	10	Maximum delivery attempts for a callback before it is abandoned.
daemon_callback_retry_base_delay_secs	--callback-retry-base-delay-secs	int	2	Initial delay between callback retries (exponential backoff, capped at 10 minutes).
daemon_webhook_callback_retry_hours	--callback-retry-ttl-hours	int	24	How long undelivered/failed webhook records are retained before cleanup.

These variables can be set in a .env file or exported in the environment (see Configuration); the CLI flags take precedence when both are set.

Webhook API

Method & Path	Description
POST /webhook/:key	Trigger analysis for a repository.
GET /health	Liveness check; returns {"status":"ok"}.

POST /webhook/:key

Trigger analysis for a repository. Only short repository names (e.g. my-repo) are accepted; full names such as org/my-repo are rejected with 400 Bad Request.

The :key path parameter is the shared secret configured via daemon_webhook_key (or --webhook-key). It authenticates the caller; a request with the wrong key is rejected with 401 Unauthorized.

Request

Send a JSON body with Content-Type: application/json. The only required field is repository.

{
  "repository": "my-repo",
  "callback_url": "https://your-app.example.com/callbacks/repo-tagger"
}

Field	Type	Required	Description
repository	string	✅	Short repository name (e.g. my-repo). Full names like org/my-repo are rejected.
callback_url	string	–	Optional HTTP(S) URL that receives the callback payload after analysis completes. Only http and https schemes are accepted.

Example requests

Trigger analysis without a callback — the results are still stored and reported, but nothing is POSTed back to you:

curl -X POST "http://localhost:3000/webhook/your-secret-key" \
  -H "Content-Type: application/json" \
  -d '{"repository": "my-repo"}'

Trigger analysis with a callback URL:

curl -X POST "http://localhost:3000/webhook/your-secret-key" \
  -H "Content-Type: application/json" \
  -d '{"repository": "my-repo", "callback_url": "https://your-app.example.com/callbacks/repo-tagger"}'

Responses

The endpoint acknowledges the request immediately (typically < 5ms) with 202 Accepted and returns before analysis runs. It does not hold the connection open while the LLM processes the repository; results arrive later via the callback_url.

Status	Meaning
202 Accepted	Request recorded and queued. Body: {"status":"accepted","message":"Repository queued for processing"}. Subsequent requests for an already-queued or in-flight repository are merged and also acknowledged with 202.
400 Bad Request	Invalid payload — an empty repository or a full name like org/my-repo. Body: {"status":"error","message":"..."}.
401 Unauthorized	Invalid webhook key. Body: {"status":"error","message":"Invalid webhook key"}.
422 Unprocessable Entity	The JSON body could not be deserialized — malformed JSON or an invalid field (e.g. a callback_url that is not a valid HTTP(S) URL).

Success response body:

{
  "status": "accepted",
  "message": "Repository queued for processing"
}

Error response body:

{
  "status": "error",
  "message": "Human-readable error detail"
}

GET /health

curl "http://localhost:3000/health"

{ "status": "ok" }

Callback Payload

To receive results asynchronously, include a callback_url in your request. When analysis completes successfully, the daemon POSTs the following JSON to that URL:

{
  "repository": "my-repo",
  "url": "https://git.yourdomain.com/org/my-repo",
  "description": "Short LLM-generated summary of the repository.",
  "topics": ["cve-2024-1234", "linux", "rust"]
}

Field	Type	Description
repository	string	Short repository name (e.g. my-repo).
url	string	Full URL of the repository page on the Forgejo instance.
description	string	Short free-text summary of the repository produced by the LLM.
topics	string[]	The labels/topics assigned to the repository.

The callback is POSTed with Content-Type: application/json. It is not signed or authenticated, so use only HTTPS callback URLs you control and treat the payload as untrusted input.

Delivery & retry

Delivery is best-effort with retries:

• On any non-2xx response or network error, the daemon retries with exponential backoff: starting at daemon_callback_retry_base_delay_secs (default 2s), doubling each attempt, capped at 10 minutes.
• Up to daemon_callback_max_retries (default 10) attempts are made; the callback is abandoned only once the retry budget is exhausted.
• Callbacks are scheduled on an in-memory DelayQueue with their exact backoff deadline, so there is no polling. On daemon restart, undelivered callbacks are recovered from SQLite and rescheduled with the remaining backoff; records are cleaned up after daemon_webhook_callback_retry_hours (default 24).
• A callback counts as delivered only when the receiving server responds with a 2xx status.

Because a retry re-POSTs an identical payload, your handler should be idempotent — for example, ignore a callback for a repository you have already recorded, or upsert keyed on repository.

Minimal receiving endpoint

A tiny Python (Flask) example that logs and acknowledges an incoming callback:

from flask import Flask, request

app = Flask(__name__)

@app.post("/callbacks/repo-tagger")
def callback():
    payload = request.get_json()
    print(f"{payload['repository']}: {payload['description']}")
    return "", 204  # any 2xx acknowledges delivery

Failure behavior

If analysis itself fails (rather than the callback), the webhook request is recorded as failed and no callback is sent — a callback_url only fires for successful analyses.

Triggering from Forgejo

The webhook endpoint expects the custom request body shown in the Webhook API section, not Forgejo's native event payload. Forgejo sends a push event whose repository field is an object (repository.full_name, repository.name, …), whereas repo-tagger expects repository to be a plain short-name string. To trigger analysis from Forgejo, adapt the event to the expected format with a small relay.

A minimal relay using Python:

import json
import urllib.request

def handle_forgejo_event(forgejo_event: dict, webhook_key: str):
    repo_name = forgejo_event["repository"]["name"]  # short name, no org prefix
    body = json.dumps({"repository": repo_name}).encode()
    req = urllib.request.Request(
        f"http://localhost:3000/webhook/{webhook_key}",
        data=body,
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    urllib.request.urlopen(req)

Point the Forgejo webhook's Target URL at the relay instead of the daemon directly. The relay passes the short repository name to the daemon.

If you control the client, you can post the custom request body directly to the daemon without a relay.

Behavior Details

• Instant webhook response: Webhooks are recorded in SQLite and enqueued in-memory, then acknowledged with 202 Accepted immediately — no HTTP connection is held open during LLM analysis.
• Deduplication: Webhook and periodic requests for the same repository merge into a single queue item; high-priority webhook arrivals promote an already-queued periodic scan.
• Restart recovery: On startup, the daemon re-enqueues webhook requests left in Pending or Processing state. Records in Failed state (permanent analysis errors) are not re-run; they are cleaned up after daemon_webhook_callback_retry_hours.
• Callback retry: Completed-but-undelivered callbacks are retried in-memory via an event-driven DelayQueue with exponential backoff, respecting daemon_callback_max_retries.
• Periodic scanning: The scanner skips repositories whose HEAD commit is unchanged and honours the --min-analysis-interval cooldown between re-analyses.

The exact request and response schemas are available in machine-readable form as JSON Schema: docs/webhook-schemas.json.

Label Registry (Taxonomy)

The label registry is a versioned JSON file (data/labels.json) that defines the taxonomy questions the LLM must answer. It supports three category types:

• WithExamples (with explanations) — e.g. OS category: the LLM picks from known labels and can propose new ones with a textual explanation.
• WithExamples (without explanations) — e.g. CVEs: the LLM picks from a list of known CVEs or proposes new CVE identifiers.
• OpenEnd — e.g. summary: the LLM writes free-form text.

Built-in Categories

Key	Question	Type
summary	Summarize the repo in 2–4 sentences	OpenEnd
cves	Which CVEs is this repo related to?	Without explanations
software_product	Names of vulnerable software products	Without explanations
attack_name	Unique attack names	Without explanations
os	Target operating system	With explanations
protocol	Network protocols exploited	With explanations
file_format	File formats exploited	With explanations
attack_vector	Required access level (physical / local / remote)	With explanations

Submit Modes

When the LLM returns a result, labels are handled according to the --auto-accept-labels mode:

Mode	Behavior
auto-confirm	New labels are immediately added to the baseline with Confirmed status
auto-accept	New labels are added with AutoAccepted status
queue-proposal	New labels are stored in a separate proposals.json for manual review
dry-run	Labels are parsed and returned but not persisted

Categories with no_auto_submit: true (e.g. CVEs) automatically downgrade to queue-proposal regardless of the chosen mode.

Architecture

The binary has two modes: one-shot CLI commands and a long-lived daemon. Both share AppContext and the LLM Agent.

┌─────────────────────────────────────────────────────────────┐
│                     repo-tagger CLI mode                     │
│  clap ─► Commands ─► execute_*() functions                   │
│    │                                                        │
│    ▼                                                        │
│  AppContext                                                  │
│    • Config      — environment variables                     │
│    • Forgejo     — HTTP client with retry middleware        │
│    • StateStore  — SQLite (sqlx)                            │
│    • Registry    — label taxonomy (JSON)                    │
│    │                                                        │
│    ▼                                                        │
│  LLM Agent (Rig + Ollama)                                   │
│    • build_system_prompt() — renders registry               │
│    • build_user_message() — repo metadata                   │
│    • Tools: list_directory, read_file                       │
│    • Retry loop (up to 10 attempts)                         │
│    • Schema validation (jsonschema)                         │
│    • Registry.submit_result() — persists labels             │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│                    daemon mode (`daemon`)                    │
│                                                             │
│  axum HTTP server                                            │
│    • POST /webhook/:key — verify key (401), accept only      │
│      short repo names (400), record in SQLite, enqueue       │
│      at high priority, reply 202 immediately                │
│    • GET /health — liveness check                            │
│            │                                                │
│            ▼                                                │
│  Queue (in-memory, priority + dedup)                        │
│    • a webhook for a repo already queued by the periodic     │
│      scanner promotes it to high priority; duplicates merge  │
│            │                                                │
│            ▼                                                │
│  Processor                                                    │
│    • marks webhook requests as Processing (SQLite)           │
│    • runs analysis via AppContext + LLM Agent                │
│    • builds CallbackPayload (short repo name + full URL)     │
│    • persists results and spawns callback delivery           │
│            │                                                │
│            ▼                                                │
│  Callback delivery + retries                                 │
│    • POSTs the CallbackPayload to the supplied callback_url  │
│    • exponential backoff with jitter (max 10 attempts)       │
│    • background retry loop every ~5 min; 24h record TTL      │
│    • startup recovery re-enqueues Pending/Processing         │
└─────────────────────────────────────────────────────────────┘

Files stored under data/:

• state.db — SQLite database tracking per-repo state (commit hash, status, labels) and webhook requests (status, callback payload, delivery/retry state).
• labels.json — The label registry taxonomy (versioned, baseline + auto-confirmed labels).
• proposals.json — Pending label proposals awaiting review.
• analyses/*.json — Raw LLM analysis outputs per repository.
• logs/repo-tagger.log — Structured log file (tracing).

Development

Prerequisites

• Rust stable toolchain (rustup install stable && rustup default stable)
• A running Ollama instance (for end-to-end testing) or use --apply-mode dry-run

Setup

git clone <repo-url>
cd repo-tagger
cp .env.example .env   # fill in your Forgejo credentials
cargo build
cargo run -- list       # verify connectivity

Testing

Run the full test suite:

cargo test

Run tests with a single thread (recommended when tests share a .env file or database path):

cargo test -- --test-threads=1

Run a specific test:

cargo test test_execute_process_dry_run

The test suite uses:

• wiremock — HTTP mock server to simulate Forgejo API responses.
• tempfile — Temporary directories and databases, cleaned up automatically.
• FakeLlmClient — In-memory mock LLM client that returns deterministic responses, avoiding any real Ollama calls.

Coverage Reports

cargo-tarpaulin

cargo install cargo-tarpaulin
cargo tarpaulin --ignore-tests

Generate an HTML report:

cargo tarpaulin --out Html --output-dir coverage

cargo-llvm-cov

cargo install cargo-llvm-cov
cargo llvm-cov --html

Open the HTML report in your browser:

cargo llvm-cov --open

Note

: cargo-llvm-cov may require a nightly Rust toolchain or a recent stable that supports -Cinstrument-coverage.

Benchmarks

Criterion benchmarks are included for performance-sensitive components:

cargo bench