Skip to main content

Why Dynamic Linking?

Go’s plugin system requires dynamic linking to load .so files at runtime. By default, Bifrost builds are statically linked for maximum portability across Linux distributions - they bundle all dependencies including the C standard library (libc). However, statically linked binaries cannot load Go plugins. To use custom plugins with Bifrost, you must build a dynamically linked binary that links against the system’s libc at runtime.
Dynamic plugins only work on Linux and macOS (Darwin). Windows is not supported by Go’s plugin system.

Static vs Dynamic Builds

Static Builds (Default)

Bifrost’s default build configuration creates statically linked binaries:
go build \
  -ldflags="-w -s -extldflags '-static' -X main.Version=v1.3.30" \
  -tags "sqlite_static" \
  -o bifrost-http
Characteristics:
  • ✅ Portable across all Linux distributions (musl, glibc, etc.)
  • ✅ No external dependencies required at runtime
  • ✅ Smaller deployment surface area
  • Cannot load Go plugins
Use static builds when: You don’t need custom plugins and want maximum portability.

Dynamic Builds (For Plugins)

To enable plugin support, build without static linking flags:
go build \
  -ldflags="-w -s -X main.Version=v1.3.30" \
  -o bifrost-http
Characteristics:
  • Can load Go plugins (.so files)
  • ✅ Slightly faster compilation
  • ⚠️ Must match the target system’s libc (musl vs glibc)
  • ⚠️ Less portable across different Linux distributions
Use dynamic builds when: You need custom plugin support.

Building with Makefile

The easiest way to build a dynamic binary is using the DYNAMIC=1 flag with the Makefile:

Local Build

# Build dynamically linked binary for your current platform
make build DYNAMIC=1

# With version tag
make build DYNAMIC=1 VERSION=1.3.30
This creates tmp/bifrost-http as a dynamically linked binary.

Cross-Compilation

# Build for Linux AMD64 (uses Docker if cross-compiling)
make build DYNAMIC=1 GOOS=linux GOARCH=amd64

# Build for Linux ARM64
make build DYNAMIC=1 GOOS=linux GOARCH=arm64

How It Works

The DYNAMIC=1 flag automatically:
  • ✅ Removes -extldflags "-static" from ldflags
  • ✅ Removes -tags "sqlite_static" build tag
  • ✅ Keeps CGO_ENABLED=1 (required for SQLite and plugins)
  • ✅ Uses Docker for cross-compilation when needed

Building with Docker

For containerized deployments, you’ll need to modify the Dockerfile. Here are two complete examples based on your target environment’s libc.

Option A: Alpine Linux (musl libc)

Use this for Alpine-based deployments or when you want minimal image size.
# --- UI Build Stage: Build the Next.js frontend ---
FROM node:24-alpine3.22 AS ui-builder
WORKDIR /app

# Copy UI package files and install dependencies
COPY ui/package*.json ./
RUN npm ci

# Copy UI source code
COPY ui/ ./

# Build UI (skip the copy-build step)
RUN npx next build
RUN node scripts/fix-paths.js

# --- Go Build Stage: Compile the Go binary ---
FROM golang:1.24.3-alpine3.22 AS builder
WORKDIR /app

# Install dependencies including gcc for CGO and sqlite
RUN apk add --no-cache gcc musl-dev sqlite-dev

# Set environment for CGO-enabled build (required for go-sqlite3 and plugins)
ENV CGO_ENABLED=1 GOOS=linux

COPY transports/go.mod transports/go.sum ./
RUN go mod download

# Copy source code and dependencies
COPY transports/ ./

COPY --from=ui-builder /app/out ./bifrost-http/ui

# Build the binary with CGO enabled for DYNAMIC LINKING
ENV GOWORK=off
ARG VERSION=unknown
RUN go build \
    -ldflags="-w -s -X main.Version=v${VERSION}" \
    -a -trimpath \
    -o /app/main \
    ./bifrost-http

# Verify build succeeded
RUN test -f /app/main || (echo "Build failed" && exit 1)

# --- Runtime Stage: Minimal runtime image ---
FROM alpine:3.22
WORKDIR /app

# Install runtime dependencies for CGO-enabled dynamic binary
# musl: C standard library (required for CGO binaries)
# libgcc: GCC runtime library
# ca-certificates: For HTTPS connections
RUN apk add --no-cache musl libgcc ca-certificates

# Create data directory and set up user
COPY --from=builder /app/main .
COPY --from=builder /app/docker-entrypoint.sh .

# Getting arguments
ARG ARG_APP_PORT=8080
ARG ARG_APP_HOST=0.0.0.0
ARG ARG_LOG_LEVEL=info
ARG ARG_LOG_STYLE=json
ARG ARG_APP_DIR=/app/data

# Environment variables with defaults (can be overridden at runtime)
ENV APP_PORT=$ARG_APP_PORT \
    APP_HOST=$ARG_APP_HOST \
    LOG_LEVEL=$ARG_LOG_LEVEL \
    LOG_STYLE=$ARG_LOG_STYLE \
    APP_DIR=$ARG_APP_DIR

RUN mkdir -p $APP_DIR/logs && \
    adduser -D -s /bin/sh appuser && \
    chown -R appuser:appuser /app && \
    chmod +x /app/docker-entrypoint.sh
USER appuser

# Declare volume for data persistence
VOLUME ["/app/data"]
EXPOSE $APP_PORT

# Health check for container status monitoring
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD wget --no-verbose --tries=1 --spider http://127.0.0.1:${APP_PORT}/metrics || exit 1

# Use entrypoint script that handles volume permissions and argument processing
ENTRYPOINT ["/app/docker-entrypoint.sh"]
CMD ["/app/main"]
Key changes from static build:
  • Line 40-44: Removed -extldflags '-static' and -tags "sqlite_static"
  • Removed UPX compression step (optional, but simpler)
  • Runtime uses musl libc from Alpine base image
Build and run:
# Build the image
docker build -f transports/Dockerfile -t bifrost:dynamic-alpine .

# Run the container
docker run -p 8080:8080 -v ./plugins:/app/data/plugins bifrost:dynamic-alpine

Option B: Debian (glibc)

Use this for Debian/Ubuntu-based deployments or when deploying to glibc-based systems.
# --- UI Build Stage: Build the Next.js frontend ---
FROM node:24-bookworm AS ui-builder
WORKDIR /app

# Copy UI package files and install dependencies
COPY ui/package*.json ./
RUN npm ci

# Copy UI source code
COPY ui/ ./

# Build UI
RUN npx next build
RUN node scripts/fix-paths.js

# --- Go Build Stage: Compile the Go binary ---
FROM golang:1.24.3-bookworm AS builder
WORKDIR /app

# Install dependencies including gcc for CGO and sqlite
RUN apt-get update && apt-get install -y \
    gcc \
    libc6-dev \
    libsqlite3-dev \
    && rm -rf /var/lib/apt/lists/*

# Set environment for CGO-enabled build (required for go-sqlite3 and plugins)
ENV CGO_ENABLED=1 GOOS=linux

COPY transports/go.mod transports/go.sum ./
RUN go mod download

# Copy source code and dependencies
COPY transports/ ./

COPY --from=ui-builder /app/out ./bifrost-http/ui

# Build the binary with CGO enabled for DYNAMIC LINKING
ENV GOWORK=off
ARG VERSION=unknown
RUN go build \
    -ldflags="-w -s -X main.Version=v${VERSION}" \
    -a -trimpath \
    -o /app/main \
    ./bifrost-http

# Verify build succeeded
RUN test -f /app/main || (echo "Build failed" && exit 1)

# --- Runtime Stage: Minimal runtime image ---
FROM debian:bookworm-slim
WORKDIR /app

# Install runtime dependencies for CGO-enabled dynamic binary
# libc6: GNU C Library (required for glibc-linked binaries)
# ca-certificates: For HTTPS connections
RUN apt-get update && apt-get install -y \
    libc6 \
    ca-certificates \
    wget \
    && rm -rf /var/lib/apt/lists/*

# Create data directory and set up user
COPY --from=builder /app/main .
COPY --from=builder /app/docker-entrypoint.sh .

# Getting arguments
ARG ARG_APP_PORT=8080
ARG ARG_APP_HOST=0.0.0.0
ARG ARG_LOG_LEVEL=info
ARG ARG_LOG_STYLE=json
ARG ARG_APP_DIR=/app/data

# Environment variables with defaults (can be overridden at runtime)
ENV APP_PORT=$ARG_APP_PORT \
    APP_HOST=$ARG_APP_HOST \
    LOG_LEVEL=$ARG_LOG_LEVEL \
    LOG_STYLE=$ARG_LOG_STYLE \
    APP_DIR=$ARG_APP_DIR

RUN mkdir -p $APP_DIR/logs && \
    useradd -m -s /bin/sh appuser && \
    chown -R appuser:appuser /app && \
    chmod +x /app/docker-entrypoint.sh
USER appuser

# Declare volume for data persistence
VOLUME ["/app/data"]
EXPOSE $APP_PORT

# Health check for container status monitoring
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD wget --no-verbose --tries=1 --spider http://127.0.0.1:${APP_PORT}/metrics || exit 1

# Use entrypoint script that handles volume permissions and argument processing
ENTRYPOINT ["/app/docker-entrypoint.sh"]
CMD ["/app/main"]
Key differences from Alpine version:
  • Uses bookworm (Debian 12) base images instead of Alpine
  • Installs apt packages instead of apk
  • Runtime uses glibc (libc6) instead of musl
  • Uses useradd instead of adduser for user creation
Build and run:
# Build the image
docker build -f transports/Dockerfile.debian -t bifrost:dynamic-debian .

# Run the container
docker run -p 8080:8080 -v ./plugins:/app/data/plugins bifrost:dynamic-debian

libc Compatibility

Understanding libc (C standard library) compatibility is critical when building dynamic binaries and plugins.

musl vs glibc

Linux distributions use one of two main C standard libraries:
libc TypeUsed ByCharacteristics
muslAlpine LinuxLightweight, minimal, security-focused
glibcDebian, Ubuntu, RHEL, CentOS, Fedora, Amazon LinuxStandard GNU C Library, feature-rich

The Golden Rule

  • A binary built with musl will NOT run on glibc systems.
  • A binary built with glibc will NOT run on musl systems.
  • Plugins and Bifrost MUST use the same libc.

Why This Matters

When you build a dynamic binary:
# Built on Alpine (musl)
$ ldd bifrost-http
        linux-vdso.so.1
        libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1

# Built on Debian (glibc)
$ ldd bifrost-http
        linux-vdso.so.1
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
The binary is linked to a specific libc implementation. If you try to run it on a system with a different libc, you’ll get errors like:
error while loading shared libraries: libc.musl-x86_64.so.1: cannot open shared object file

Choosing Your Build Environment

Decision Matrix:
Target DeploymentBuild WithDockerfile Base
Alpine containersmuslgolang:1.24.3-alpine3.22
Debian/Ubuntu containersglibcgolang:1.24.3-bookworm
Ubuntu/Debian serversglibcgolang:1.24.3-bookworm
RHEL/CentOS serversglibcNative build or glibc container
Kubernetes (Alpine)muslgolang:1.24.3-alpine3.22
Kubernetes (Debian)glibcgolang:1.24.3-bookworm
Simple rule: Build with the same base OS family as your deployment target.

Building Plugins

Plugins must be built with the exact same environment as your Bifrost binary:
# If Bifrost was built with Alpine/musl
docker run --rm \
  -v "$PWD:/work" \
  -w /work \
  golang:1.24.3-alpine3.22 \
  sh -c "apk add --no-cache gcc musl-dev && \
         go build -buildmode=plugin -o myplugin.so main.go"

# If Bifrost was built with Debian/glibc  
docker run --rm \
  -v "$PWD:/work" \
  -w /work \
  golang:1.24.3-bookworm \
  sh -c "apt-get update && apt-get install -y gcc && \
         go build -buildmode=plugin -o myplugin.so main.go"
See the hello-world plugin Makefile for a complete example.

Verification

Verify Dynamic Linking

After building, check that your binary is dynamically linked:
# Check binary dependencies
ldd tmp/bifrost-http

# Expected output (musl):
linux-vdso.so.1
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1

# Expected output (glibc):
linux-vdso.so.1
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
If you see statically linked, the binary will not load plugins.

Verify Plugin Compatibility

Test that your plugin loads successfully:
# Start Bifrost with your plugin configured
./tmp/bifrost-http -config config.json

# Check logs for plugin initialization
# Should see: "Plugin loaded successfully: your-plugin-name"

Go Version and Package Compatibility

Go Version Requirement

Bifrost is built with Go 1.24.3. Your plugin must be compiled with the exact same Go version to ensure compatibility.
# Check your Go version
go version
# Should output: go version go1.24.3 ...

# If you need to install Go 1.24.3
# Visit: https://go.dev/dl/

Key Package Versions

Bifrost uses the following key packages across its three main modules that may affect plugin development:

Transport Layer (transports/go.mod)

PackageVersionPurpose
github.com/bytedance/sonicv1.14.1High-performance JSON serialization
github.com/valyala/fasthttpv1.67.0Fast HTTP server/client
github.com/fasthttp/routerv1.5.4HTTP router for fasthttp
github.com/fasthttp/websocketv1.5.12WebSocket support
github.com/prometheus/client_golangv1.23.0Prometheus metrics
gorm.io/gormv1.31.1Database ORM

Core Layer (core/go.mod)

PackageVersionPurpose
github.com/bytedance/sonicv1.14.1High-performance JSON serialization
github.com/valyala/fasthttpv1.67.0Fast HTTP client for providers
github.com/google/uuidv1.6.0UUID generation
github.com/rs/zerologv1.34.0Zero-allocation JSON logger
github.com/mark3labs/mcp-gov0.41.1Model Context Protocol support
golang.org/x/oauth2v0.32.0OAuth2 client

Framework Layer (framework/go.mod)

PackageVersionPurpose
github.com/redis/go-redis/v9v9.14.0Redis client for caching
github.com/weaviate/weaviate-go-client/v5v5.5.0Weaviate vector store client
github.com/mattn/go-sqlite3v1.14.32SQLite3 driver (requires CGO)
gorm.io/gormv1.31.1Database ORM
gorm.io/driver/sqlitev1.6.0GORM SQLite driver
gorm.io/driver/postgresv1.6.0GORM PostgreSQL driver
golang.org/x/cryptov0.43.0Cryptographic functions
If your plugin imports any of these packages, use compatible versions to avoid runtime issues. Check transports/go.mod, core/go.mod, and framework/go.mod for complete dependency lists.

Checking Bifrost’s Dependencies

To see all dependencies used by Bifrost across its three main modules:
# View transport layer dependencies
cat transports/go.mod

# View core dependencies
cat core/go.mod

# View framework dependencies
cat framework/go.mod

# Or list all dependencies for a specific module
cd transports && go list -m all
cd ../core && go list -m all
cd ../framework && go list -m all

Plugin go.mod Example

When creating a plugin, your go.mod should match Bifrost’s Go version:
module github.com/example/my-plugin

go 1.24.3

require (
    github.com/maximhq/bifrost/core v1.2.26
    // Optional: Add framework for advanced features
    // github.com/maximhq/bifrost/framework v1.1.33
    
    // Add other dependencies as needed, matching versions from Bifrost's go.mod files
    // github.com/bytedance/sonic v1.14.1
    // github.com/rs/zerolog v1.34.0
)
Import only the Bifrost modules you need. Most plugins only require core. Use framework if you need access to config stores, vector stores, or other framework features.

Troubleshooting

Common Errors

1. Cannot load plugin - Go version mismatch

cannot load plugin: plugin was built with a different version of package runtime/internal/sys
Cause: Plugin and Bifrost were built with different Go versions. Solution: Use the exact same Go version (Go 1.24.3) for both:
# Check Go version used for Bifrost
./tmp/bifrost-http -version

# Verify your Go version matches
go version  # Should output: go version go1.24.3

# See full compatibility requirements
Refer to Go Version and Package Compatibility for details.

2. Shared library not found

error while loading shared libraries: libc.musl-x86_64.so.1: cannot open shared object file
Cause: Binary built with musl trying to run on glibc system (or vice versa). Solution: Rebuild with the correct libc for your target system.

3. Plugin architecture mismatch

plugin was built with a different version of package internal/cpu
Cause: Plugin and Bifrost built for different architectures (amd64 vs arm64). Solution: Ensure GOARCH matches for both builds:
# Check architecture
uname -m  # x86_64 = amd64, aarch64 = arm64

# Build with explicit architecture
GOARCH=amd64 go build ...

4. Plugin file not found

plugin.Open("myplugin.so"): realpath failed: no such file or directory
Cause: Plugin file path is incorrect in config. Solution: Use absolute paths or verify relative paths:
{
  "plugins": [
    {
      "path": "/app/data/plugins/myplugin.so",
      "config": {}
    }
  ]
}

Best Practices

1. Document Your Build Environment

Create a BUILD.md file documenting:
  • Go version used
  • Base image (Alpine vs Debian)
  • Build commands
  • Target deployment platform

2. Use Consistent Tooling

Match Bifrost’s exact Go version and key dependencies (see Go Version and Package Compatibility):
# Pin Go version in Dockerfile
FROM golang:1.24.3-alpine3.22 AS builder

# Pin Go version in Makefile/CI
GO_VERSION=1.24.3

3. Test Plugin Loading Locally

Before deploying, test plugin loading:
# Build both Bifrost and plugin
make build DYNAMIC=1
cd examples/plugins/hello-world && make build

# Test loading
./tmp/bifrost-http -config examples/plugins/hello-world/config.json

4. Version Your Plugins

Tag plugin builds with version and build info:
go build -buildmode=plugin \
  -ldflags="-X main.Version=v1.0.0 -X main.BuildTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  -o myplugin-v1.0.0.so

5. Multi-Stage Dockerfiles for Plugins

Build plugins in the same Dockerfile as Bifrost:
# Build plugin
FROM golang:1.24.3-alpine3.22 AS plugin-builder
WORKDIR /plugin
COPY plugins/myplugin/ .
RUN apk add --no-cache gcc musl-dev && \
    go build -buildmode=plugin -o myplugin.so main.go

# Build Bifrost
FROM golang:1.24.3-alpine3.22 AS bifrost-builder
# ... (bifrost build steps)

# Runtime
FROM alpine:3.22
COPY --from=bifrost-builder /app/main .
COPY --from=plugin-builder /plugin/myplugin.so /app/plugins/
This ensures plugins and Bifrost use identical build environments.

Next Steps

Now that you have a dynamically linked Bifrost binary:
  1. Write your first plugin - Learn the plugin API and create custom functionality
  2. Deploy with plugins - Best practices for production deployments
  3. Example plugins - Study working examples
For questions or issues with dynamic builds and plugins, visit our GitHub Discussions or Discord community.