Skip to content

Docker Packaging Guide

This guide explains how to build, run, and optimize the GoServe Docker image.

Step-by-Step: From Zero to Running

If you have just cloned the repository, follow these steps to get a containerized GoServe instance running with a sample model.

1. Clone the Repository

git clone https://github.com/goserve-run/goserve.git
cd goserve

2. Build the Docker Image

No local dependencies (like Go or ONNX) are needed on your host machine; the Dockerfile handles everything. 

docker build -t goserve:latest .

3. Run the Container

We will mount the examples folder so we can test with the image classification model. 

# Using absolute path for volume mounting (recommended)
docker run -d \
  --name goserve \
  -p 8080:8080 \
  -v "$(pwd)/examples:/app/examples" \
  goserve:latest

4. Load the Model & Test

# Load model
curl -X POST http://localhost:8080/v1/models \
  -H "Content-Type: application/json" \
  -d '{"name": "mobilenet", "path": "/app/examples/image-classification/models/mobilenetv2.onnx"}'

# Check health
curl http://localhost:8080/health

How the Multi-Stage Build Works

GoServe uses a Multi-Stage Dockerfile. This is a professional pattern that separates the build environment from the runtime environment.

Why use Multi-Stage?

If we used a single stage, our final image would be >1GB because it would contain the Go compiler, C++ build tools (gcc), and source code. By using two stages, our final image is under 100MB and only contains the bare essentials needed to run.

Stage 1: The Builder (AS builder)

  • Base: golang:1.25-bookworm (A full Go development environment).
  • Setup: We install build-essential so we can use CGO (needed for ONNX).
  • Dependency: We download the ONNX Runtime Linux x64 shared library (.so) version 1.23.2 from GitHub.
  • Compilation: We build the Go binary. We use CGO_LDFLAGS and CGO_CFLAGS to tell the Go compiler exactly where we put the ONNX library files.
  • Result: We have a compiled binary named goserve and the library libonnxruntime.so.

Stage 2: The Runtime (Final Image)

  • Base: gcr.io/distroless/cc-debian12 (A tiny image with only C++ runtime libraries).
  • Cleanup: We throw away the Go compiler, the source code, and all the build tools.
  • Copying: We only copy two things from the builder stage:
    1. The goserve binary.
    2. The libonnxruntime.so library (placed into /usr/lib).
  • Execution: We set LD_LIBRARY_PATH so the binary can find the ONNX library at startup.

Prerequisites

  • Docker Desktop (Windows/macOS) or Docker Engine (Linux).
  • At least 2GB of RAM available for the build process (ONNX library is large).
  • An internet connection to download dependencies during the build.

Potential Problems & Must-Haves

1. Library Path (Must-Have)

The binary requires libonnxruntime.so to be in the system's library path. Our Dockerfile handles this by copying it to /usr/lib and setting ENV LD_LIBRARY_PATH=/usr/lib.

2. Architecture Mismatch

The current Dockerfile downloads the x64 (amd64) version of ONNX Runtime. If you are building on Apple Silicon (M1/M2) or ARM64 Linux, the binary will not run. - Fix: Update the download URL in the Dockerfile to use the aarch64 package for ARM builds.

3. Permissions

On Linux, ensure the user running Docker has permissions to read the model files you are mounting. If you see "Permission Denied" in the container logs, check the UID/GID of your model folder.

Verifying the Build

We provide a cross-platform Python script to verify your Docker setup:

python scripts/verify_docker.py
This script automates the build, starts a container, and runs a test inference using the MobileNet example.