Skip to content

Quick Start & Installation

GoServe is a high-performance ML inference server. While it is built in Go, it leverages the ONNX Runtime C++ library, which requires some platform-specific setup.

Docker is the easiest way to run GoServe as it bundles all C++ dependencies and works identically on Windows, Linux, and macOS.

# 1. Clone
git clone https://github.com/goserve-run/goserve.git && cd goserve

# 2. Build
docker build -t goserve .

# 3. Run
docker run -d -p 8080:8080 -v "$(pwd)/examples:/app/examples" goserve

Limitations: - Windows: Requires Docker Desktop with WSL2 backend. - macOS: Apple Silicon (M1/M2) users may need to adjust the Dockerfile to download the aarch64 version of ONNX Runtime.

🛠️ 2. Local Installation

Building GoServe locally provides the lowest latency and easiest access to local hardware.

Prerequisites

OS Required C Compiler Library Requirement
Windows MinGW-w64 onnxruntime.dll
Linux gcc libonnxruntime.so
macOS clang (via Xcode) libonnxruntime.dylib

Step 1: Download ONNX Runtime

GoServe requires ONNX Runtime v1.23.2. Download the shared library for your platform from the official releases.

Step 2: Build the Server

Windows (PowerShell)

  1. Ensure gcc.exe from MinGW is in your PATH.
  2. Ensure onnxruntime.dll is in the same folder as the binary or in your PATH. 
    $env:CGO_ENABLED = "1"
go build -o goserve.exe ./cmd/server

Linux

  1. Install build essentials: sudo apt install build-essential.
  2. Add the library to your search path: export LD_LIBRARY_PATH=/path/to/onnx/lib. 
    export CGO_ENABLED=1
go build -o goserve ./cmd/server

macOS

  1. Install Xcode Command Line Tools: xcode-select --install.
  2. Add the library to your search path: export DYLD_LIBRARY_PATH=/path/to/onnx/lib. 
    export CGO_ENABLED=1
go build -o goserve ./cmd/server

Platform Limitations

Windows

  • CGO Complexity: You must use a gcc compiler that matches the architecture (x64) of your Go installation. We recommend WinLibs.
  • DLL Path: If you see "Library not found," ensure onnxruntime.dll is in your project root.

Linux

  • Glibc Version: The pre-built ONNX binaries require a relatively modern glibc. If you are on an extremely old distro (e.g., CentOS 7), you may need to compile ONNX from source.

macOS

  • Code Signing: Newer versions of macOS may block the ONNX library because it is "unsigned." You might need to manually allow it in System Settings -> Privacy & Security.
  • Architecture: Ensure you download the universal or arm64 build for Apple Silicon.

🧠 3. Verify Installation

Once built or running, load the MobileNet example:

# Load model
curl -X POST http://localhost:8080/v1/models \
  -H "Content-Type: application/json" \
  -d '{"name": "mobilenet", "path": "/app/examples/image-classification/models/mobilenetv2.onnx"}'

# Check metrics
curl http://localhost:8080/metrics