Import Bot

import-bot is a Matrix bot written in Rust. It monitors Matrix rooms for GitHub repository links, checks if they have already been cloned, retrieves metadata (such as repository size) from GitHub, and allows authorized users to clone/migrate them directly to a self-hosted Forgejo instance simply by reacting with a thumbs-up (👍️).

---

Features

• Automated Link Detection: Scans incoming text messages (both plain text and HTML formats) for GitHub repository links.
• Duplicate Prevention: Queries the Forgejo API to check whether a repository has already been cloned.
• Repository Size Metadata: Fetches the repository size from the GitHub API and displays it formatted for human readability (e.g., 12.5 MB).
• Interactive Cloning Workflow: Posts an interactive offer message to the room. When a whitelisted user reacts to the message with 👍️, the bot automatically triggers a repository migration to Forgejo.
• Direct Cloning Command: Authorized users can clone repositories immediately by sending !clone <github-url> in any room, bypassing the reaction-based workflow.
• Dynamic Access Control: Features a file-backed user whitelist. Authorized users can dynamically whitelist other users directly inside the Matrix room.
• Secure Persistence: Automatically handles session encryption and local persistent storage for the Matrix client state and the user whitelist.
• Repository Analysis (optional): After a successful clone, optionally triggers repo-tagger to analyze the repository. An axum callback server receives the results (summary + topic tags) and posts them back to the Matrix room.

---

How It Works

1. Detection: You paste a GitHub link (e.g., https://github.com/rust-lang/rust) in a room where the bot is present.

2. Analysis: The bot checks if this repository is already cloned to the target organization on your Forgejo server.

   • If already cloned, it replies indicating where the repository can be found.
   • If not cloned, it fetches the repository size from GitHub and replies with a prompt:

     Would you like me to clone rust-lang/rust (150 MB)?
     Please react to this message with thumbs up 👍️

3. Execution (Interactive): An authorized user reacts to the prompt with 👍️. The bot triggers the Forgejo repository migration API and posts updates in-room.

4. Direct Cloning: Alternatively, whitelisted users can clone a repository directly by posting:

   !clone https://github.com/owner/repository
   

   The bot will immediately begin the cloning process and post updates in the room.

The migrated repository is named following the format {github_owner}-{github_repo} on your Forgejo instance.

---

Configuration

The bot is configured via environment variables. Because the application deserializes configuration directly using lowercase field names, these variables must be defined in lowercase (e.g., inside a .env file in the project's root directory).

Environment Variable	Description
access_token	The Matrix access token for the bot's account.
server	The URL of the Matrix homeserver (e.g., https://matrix.org).
user_name	The full MXID of the bot (e.g., @importbot:matrix.org).
recovery_passphrase	Passphrase used to back up and recover the Matrix session encryption keys.
storage	Directory path where database, session data, and whitelist files will be stored.
session_encryption_key	A 64-character hex-encoded string used to encrypt local database sessions.
device_id	The designated Device ID for the bot session.
initial_whitelisted_user	The MXID of the first admin/user who is authorized to interact with the bot and run commands (e.g., @admin:matrix.org).
forgejo_token	An API/Access token for your Forgejo instance with repository migration privileges.
forgejo_url	The base URL of your Forgejo instance (e.g., https://git.example.com).
forgejo_organization	The organization name on Forgejo where the cloned repositories will be housed.

Repository Analysis (optional)

After a repository has been successfully cloned, the bot can optionally trigger an external repo-tagger service to analyze the repository and assign topics/labels. To enable this, set analysis_enabled=true and configure the following additional variables:

Environment Variable	Description
analysis_enabled	Set to true to enable the analysis hook (default: disabled).
analysis_service_url	Full URL to the repo-tagger webhook endpoint, including its webhook key (e.g. http://repo-tagger:3000/webhook/secret-key).
analysis_callback_base	Base URL that repo-tagger can reach this bot on (e.g. http://import-bot:8080). The callback path /callback/<hmac>/<room_id> is appended automatically.
analysis_callback_secret	Shared secret used to sign the callback URL with HMAC-SHA256. Only callbacks carrying a valid HMAC are accepted.
analysis_listen_addr	Address and port for the axum callback server to listen on (e.g. 0.0.0.0:8080).

When a clone succeeds and analysis is enabled:

1. The bot sends a POST to analysis_service_url with the repository name and a signed callback URL.
2. The repo-tagger acknowledges immediately (202 Accepted) and analyzes asynchronously.
3. When done, repo-tagger POSTs back to /callback/<hmac>/<room_id> with a summary and topic tags.
4. The axum server verifies the HMAC, looks up the Matrix room, and posts the result:

   Analysis of <url> completed: <summary>
   
   #tag1 #tag2 #tag3
   

All five variables must be set when analysis_enabled=true; if any are missing, the feature silently disables itself.

Example .env File

server=https://matrix.example.com
user_name=@import-bot:example.com
access_token=syt_aBcDeF...
device_id=importbot
recovery_passphrase=a-very-secure-passphrase-here
session_encryption_key=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
storage=./storage

initial_whitelisted_user=@your-username:example.com

forgejo_url=https://forgejo.example.com
forgejo_token=gft_yourforgejoapitokenhere
forgejo_organization=mirrors

# Optional: enable repo analysis after cloning (requires all five)
# analysis_enabled=true
# analysis_service_url=http://repo-tagger:3000/webhook/secret-key
# analysis_callback_base=http://import-bot:8080
# analysis_callback_secret=your-callback-secret
# analysis_listen_addr=0.0.0.0:8080

Getting the access token

There is a python script get_token.py in the utils folder. You can run it with your Matrix server as an argument, like:

python3 get_token.py  https://matrix.milliways.info/

Then login with your browser and when done, it will print out the access token.

---

Installation & Running

Prerequisites

• Rust toolchain (Stable)
• A registered Matrix account for your bot
• A target organization and API token on a Forgejo instance

1. Build the Binary

Clone the repository and compile the project using Cargo:

cargo build --release

2. Run the Bot

Make sure your environment variables are set up (either in your environment or via a .env file), then start the application:

cargo run --release

If you encounter session issues or encryption sync errors on startup, you can temporarily run the bot with the --reset flag to clear the local session encryption state and log in fresh:

cargo run --release -- --reset

---

Management & Commands

Whitelisting Users

Only users on the whitelist are allowed to react to the bot's prompts to trigger a repository clone.

To add a new user to the whitelist, a currently whitelisted user can post the following command in any room the bot is in:

!whitelist @username:example.com

The whitelist is persisted locally in the path specified by your storage variable inside a whitelist.json file. This file can also be hand-edited (as a JSON array of strings) if needed while the bot is shut down.

Note: The bot parses user mentions using Matrix's HTML format. Ensure you mention the user properly inside the Matrix client so it is formatted as an HTML link anchor (e.g., <a href="https://matrix.to/#/@username:example.com">...</a>).

Cloning Repositories

Authorized users can clone a GitHub repository directly to your Forgejo instance by posting the following command in any room the bot is in:

!clone https://github.com/owner/repository

Requirements:

• The user must be on the whitelist (see above).
• Only one repository link is allowed per command.
• The bot will post updates in the room as the cloning progresses.

---

Development & Testing

Unit tests for GitHub link parsing, Forgejo cloning interactions, and whitelist persistence are included. You can execute them via:

cargo test

The tests mock both GitHub (using octocrab & wiremock) and Forgejo integrations, ensuring no live API requests are made during test execution.