Go with Portage and Crossdev, for easy static multi-platform compilation of CGO_ENABLED software.
Wednesday 08 November 2023 · 47 mins read · Viewed 50 timesTable of contents 🔗
Introduction 🔗
This article describes how to easily statically compile for multiple platforms. Unfortunately, it does not cover OS X, mainly because there is nothing similar to Portage on OS X. You can use OSXCross, but you'll have to compile the dependency tree manually.
Static compilation has always been difficult in C due to the fact that you have to compile everything statically, or have the statically linked libraries available on your system. This was never a pleasure, and people started jumping on the flatpak/Appimage bandwagon (which is a fair take).
However, there is a certain "sexiness" to just distribute one static binary:
- It is fully contained.
- It doesn't require an OS to run it. You can make empty Docker image with just a binary in it.
This is why, in Go, it's possible to disable CGO by setting CGO_ENABLED=0, which allows Go to compile statically.
But... what if you need CGO, because of CGO packages like sqlite and C libraries like ffmpeg? Well, let me introduce you to Portage and Crossdev.
Understanding static-linked and dynamically-linked. 🔗
Before explaining Portage and Crossdev, let me explain how C compilation and linking work.
The compilation stage is where the compiler analyzes the source code and converts it into assembly code. In the same step, the assembler included in the toolchain converts the assembly code into machine code, which is the binary file itself (often in ELF format).
1//main.c
2int main() { return 0; }
1# Compile
2gcc -S -fverbose-asm main.c -o main.s
3# Assemble
4gcc -c main.s -o main.o
5# `file` outputs "ELF 64-bit LSB relocatable"
An object file contains references to other functions and symbols defined in other object files or libraries. Furthermore, an object file contains no entry point, initialization or finalization code. It is therefore impossible to run it:
1# Inspect symbols
2readelf -s main.o
3
4# Symbol table '.symtab' contains 4 entries:
5# Num: Value Size Type Bind Vis Ndx Name
6# 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
7# 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS main.c
8# 2: 0000000000000000 0 SECTION LOCAL DEFAULT 1 .text
9# 3: 0000000000000000 11 FUNC GLOBAL DEFAULT 1 main
To resolve the references, add the initialization and finalization code, we need to use a linker like ld. This is the linking step:
1# Link
2gcc main.o -o main
3# Do "gcc main.c -o main" to do everything at once.
4# `file` outputs "ELF 64-bit LSB pie executable, dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2"
1# Inspect symbols
2readelf -s main
3
4# Symbol table '.dynsym' contains 6 entries:
5# Num: Value Size Type Bind Vis Ndx Name
6# 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
7# 1: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _[...]@GLIBC_2.34 (2)
8# 2: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _ITM_deregisterT[...]
9# 3: 0000000000000000 0 NOTYPE WEAK DEFAULT UND __gmon_start__
10# 4: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _ITM_registerTMC[...]
11# 5: 0000000000000000 0 FUNC WEAK DEFAULT UND [...]@GLIBC_2.2.5 (3)
12
13# Symbol table '.symtab' contains 23 entries:
14# Num: Value Size Type Bind Vis Ndx Name
15# 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
16# 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS main.c
17# 2: 0000000000000000 0 FILE LOCAL DEFAULT ABS
18# 3: 0000000000003e10 0 OBJECT LOCAL DEFAULT 20 _DYNAMIC
19# 4: 0000000000002004 0 NOTYPE LOCAL DEFAULT 16 __GNU_EH_FRAME_HDR
20# 5: 0000000000003fe8 0 OBJECT LOCAL DEFAULT 22 _GLOBAL_OFFSET_TABLE_
21# 6: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __libc_start_mai[...]
22# 7: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _ITM_deregisterT[...]
23# 8: 0000000000004000 0 NOTYPE WEAK DEFAULT 23 data_start
24# 9: 0000000000004010 0 NOTYPE GLOBAL DEFAULT 23 _edata
25# 10: 0000000000001160 0 FUNC GLOBAL HIDDEN 14 _fini
26# 11: 0000000000004000 0 NOTYPE GLOBAL DEFAULT 23 __data_start
27# 12: 0000000000000000 0 NOTYPE WEAK DEFAULT UND __gmon_start__
28# 13: 0000000000004008 0 OBJECT GLOBAL HIDDEN 23 __dso_handle
29# 14: 0000000000002000 4 OBJECT GLOBAL DEFAULT 15 _IO_stdin_used
30# 15: 0000000000004018 0 NOTYPE GLOBAL DEFAULT 24 _end
31# 16: 0000000000001040 34 FUNC GLOBAL DEFAULT 13 _start
32# 17: 0000000000004010 0 NOTYPE GLOBAL DEFAULT 24 __bss_start
33# 18: 0000000000001155 11 FUNC GLOBAL DEFAULT 13 main
34# 19: 0000000000004010 0 OBJECT GLOBAL HIDDEN 23 __TMC_END__
35# 20: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _ITM_registerTMC[...]
36# 21: 0000000000000000 0 FUNC WEAK DEFAULT UND __cxa_finalize@G[...]
37# 22: 0000000000001000 0 FUNC GLOBAL HIDDEN 10 _init
You can see the _init and _fini codes, which are the initialization and finalization codes respectively. You can also see that our executable is dynamically linked to GLIBC 2.34. You can also check the links with ldd:
1# Find links
2ldd main
3# linux-vdso.so.1 (0x00007fff0237e000)
4# libc.so.6 => /lib64/libc.so.6 (0x00007fd35d245000)
5# /lib64/ld-linux-x86-64.so.2 (0x00007fd35d44d000)
Now let's talk about "dynamic" and "static" linking. What we've just done is dynamic linking, i.e. we've linked external libraries (GLIBC) to our executable. The external symbols in our code will use the definition stored in external libraries (libc.so). Our binary cannot function without these external libraries. This is why each OSes need a package manager. A package manager allows everyone to use external libraries and find the right version of each one. This is also why we need to compile for every Linux operating system when we want to distribute a dynamically linked library or executable.
However, when this proves too tedious, we can try to establish a "static" link with the executable. The "static" link consists of including the compiled external symbols directly in the final object. The final object will no longer contain references, but the complete definition of the symbols:
1# Static linking (we do everything at once)
2gcc -static main.c -o main
3# `file` outputs "ELF 64-bit LSB executable, statically linked"
1# Inspect symbols
2readelf -s main
3
4# Symbol table '.symtab' contains 2015 entries:
5# Num: Value Size Type Bind Vis Ndx Name
6# 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
7# 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS libc_fatal.o
8# 2: 0000000000401100 5 FUNC LOCAL DEFAULT 6 __libc_message.cold
9# ...
The object contains no references to external symbols and is therefore no longer linked to external libraries.
1ldd main
2# not a dynamic executable
The resulting static object is also larger than the dynamic object.
However, the main reason we prefer to use a statically-linked executable is portability and ease of compilation. Statically compiled objects can run without the need for an operating system. This is an excellent choice when we want to containerize, as it allows us to build "distro-less" container images and drastically reduce the attack surface.
The linking stage is one of the steps often dreaded by programmers, which is why people tend to use pre-compiled runtimes like .NET, Go, Python and Java.
About Go compilation and CGO 🔗
When compiling a Go executable, the Go runtime is included in the final executable. If a Go dependency uses CGO, the final executable will be linked to GLIBC and the external libraries. CGO allows us to include C code in Go using the Foreign Function Interface (FFI).
github.com/mattn/go-sqlite3 is one of the most popular SQLite 3 driver for Go and uses CGO, making it hard to export a static binary.
For the sake of the example, let's use math.h, the math library of the C library.
1//main.go
2package main
3
4// We've add -lm to link against libm.so. No need to indicate the location of libm.so with -L since it is a system library.
5// No need to indicate the location of headers file too with -I.
6/*
7#cgo LDFLAGS: -lm
8#include <math.h>
9*/
10import "C"
11
12import (
13 "fmt"
14)
15
16func mySqrt(v float64) float64 {
17 return float64(C.sqrt(C.double(v)))
18}
19
20func main() {
21 fmt.Println(mySqrt(2))
22}
23
By add import "C", we are using CGO. C.sqrt is defined in the libm.so library. If we compile it:
1go build main.go
2# `file` outputs "ELF 64-bit LSB executable, dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2"
And inspect it:
1readelf -s main | less
2
3# Symbol table '.dynsym' contains 51 entries:
4# Num: Value Size Type Bind Vis Ndx Name
5# 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
6# 1: 0000000000000000 0 FUNC GLOBAL DEFAULT UND free@GLIBC_2.2.5 (2)
7# 2: 0000000000000000 0 FUNC GLOBAL DEFAULT UND [...]@GLIBC_2.3.4 (3)
8# 3: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _[...]@GLIBC_2.34 (4)
9# ...
10#
11# Symbol table '.symtab' contains 2236 entries:
12# Num: Value Size Type Bind Vis Ndx Name
13# 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
14# 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS go.go
15# 2: 0000000000402400 0 FUNC LOCAL DEFAULT 13 runtime.text
16# 3: 0000000000402400 85 FUNC LOCAL DEFAULT 13 internal/abi.Kin[...]
17# 4: 0000000000402460 22 FUNC LOCAL DEFAULT 13 internal/abi.(*T[...]
18# 5: 0000000000402480 180 FUNC LOCAL DEFAULT 13 internal/abi.(*T[...]
19# 6: 0000000000402540 68 FUNC LOCAL DEFAULT 13 internal/abi.(*T[...]
20# 7: 00000000004025a0 123 FUNC LOCAL DEFAULT 13 internal/abi.Nam[...]
21# ...
22
23ldd main
24# linux-vdso.so.1 (0x00007fff685fe000)
25# libm.so.6 => /lib64/libm.so.6 (0x00007f76e20ed000)
26# libc.so.6 => /lib64/libc.so.6 (0x00007f76e1f13000)
27# /lib64/ld-linux-x86-64.so.2 (0x00007f76e21f1000)
You can see that the executable is linked with GLIBC and contains the Go runtime symbols.
So how do we compile statically with CGO_ENABLED?
With this:
1go build -ldflags '-extldflags "-static"' main.go
However, when we compile statically, we need all dependencies to be static libraries. Here, we need libm.a, the static version of libm.so :
1find /usr -name libm.a
2# /usr/lib64/libm.a
Since it's a system library, it's easy. But what if you need a whole tree of dependencies that need to be statically compiled? Well, here's Portage.
Portage: Gentoo's package manager 🔗
As you may know, Gentoo is an operating system renowned for the fact that all packages are built at installation time. Some people may say that it is not maintainable (which I may agree), but it is certainly the one of most stable OS of all time.
This is thanks to Portage, Gentoo's package manager. Although it's called a "package manager", it's closer to a build system like Arch Build System, than a package manager like APT. Since it's a build system, we can compile and install all the dependencies we want.
For example, let's use libcurl:
1//main.go
2package main
3
4// Use pkg-config instead of LDFLAGS to resolve all the LDFLAGS.
5// You can try it on your terminal by running "pkg-config libcurl"
6/*
7#cgo pkg-config: libcurl
8#include <string.h>
9#include <curl/curl.h>
10*/
11import "C"
12
13import (
14 "fmt"
15)
16
17// strndup is used to safely convert a *C.char string into a Go string.
18func strndup(cs *C.char, len int) string {
19 return C.GoStringN(cs, C.int(C.strnlen(cs, C.size_t(len))))
20}
21
22func curlVersion() string {
23 // curl_version returns a static buffer of 300 characters.
24 return strndup(C.curl_version(), 300)
25}
26
27func main() {
28 fmt.Println(curlVersion())
29}
30
Just for the test, install libcurl-dev (whatever is the name of the cURL development package of your distribution) and test it:
1go build main.go
2./main
3# libcurl/8.4.0 OpenSSL/3.1.4 zlib/1.2.13 zstd/1.5.5 c-ares/1.19.1 nghttp2/1.57.0
Now let's do it with Portage. To customize software, there's usually a configuration stage by using CMake, automake, autoconf... With Portage, we manipulate USE flags to customize a package. To compile static libraries, we need to use the static-libs USE flag on our dependencies.
Let's make a Dockerfile to automate the whole process. We use the Docker image gentoo/stage3:musl (we use musl libc instead of glibc to avoid any issues with static linking):
1#Dockerfile
2FROM gentoo/stage3:musl
3
4# TODO: install Go
5# TODO: install dependencies with `static-libs`
6# TODO: statically compile our go application
Let's install Go:
1FROM gentoo/stage3:musl
2
3# Synchronize with gentoo repository
4RUN emerge --sync
5
6# Install Go dependencies
7RUN MAKEOPTS="-j$(nproc)" USE="gold" emerge sys-devel/binutils dev-vcs/git
8
9# Install Go (ACCEPT_KEYWORDS="~*" means to allows "unstable" version)
10RUN MAKEOPTS="-j$(nproc)" \
11 ACCEPT_KEYWORDS="~*" \
12 emerge ">=dev-lang/go-1.21.0"
13
14# TODO: install dependencies with `static-libs`
15# TODO: statically compile our go application
Then install the dependencies. cURL is available as net-misc/curl. From the packages.gentoo.org page, you can see the available USE flags and ebuild files, which describes the configuration and compilation steps of the package.
1#...
2
3RUN MAKEOPTS="-j$(nproc)" \
4 ACCEPT_KEYWORDS="~*" \
5 USE="static-libs" \
6 emerge --newuse \
7 net-misc/curl \
8 sys-libs/zlib \
9 net-libs/nghttp2 \
10 net-dns/c-ares # We need to add net-libs/nghttp2, sys-libs/zlib and net-dns/c-ares, because the maintainer of net-misc/curl forgot to spread "static-libs" to these packages packages
11
12# TODO: statically compile our go application
Then, let's copy our source code and statically compile our application:
1# ...
2
3COPY main.go .
4
5RUN go build -ldflags '-extldflags "-static"' -o main main.go
Let's build:
1docker build -t builder .
You will see some undefined references. This is because since we are statically compiling, we need to statically link with the dependencies of curl. Let's edit main.go:
1//main.go
2package main
3
4// HERE
5/*
6#cgo pkg-config: libcurl libssl libcrypto libnghttp2 libcares zlib
7#include <string.h>
8#include <curl/curl.h>
9*/
10import "C"
11
12// ...
Let's build again:
1docker build -t builder .
Which should work, and let's extract the executable:
1mkdir -p out
2docker run --rm \
3 -v $(pwd)/out:/out \
4 builder \
5 cp /main /out/main
Then, you can run it:
1out/main
2# libcurl/8.4.0 OpenSSL/3.1.4 zlib/1.2.13 zstd/1.5.5 c-ares/1.19.1 nghttp2/1.57.0
We've successfully compiled statically for amd64! Now let's build for multiple platforms using Crossdev!
Crossdev: Gentoo's cross-compilation environment 🔗
Let's talk about cross-compiling. Cross-compiling means compiling with a different build platform than the target platform. For example, compiling for linux/riscv64 on a linux/amd64 machine is cross-compiling, just as compiling for windows/amd64 on a linux/amd64 machine is also cross-compiling.
Cross-compiling can be achieved using either:
- A cross-compiler with its cross-compiling environment
- A virtualized environment (as with Qemu).
Using Qemu is more stable and probably a safer way of cross-compiling, but it is the slower method. Using a cross-compiler is faster because we avoid the virtualization layer.
The aim of Crossdev is to easily install a cross-compiling environment on Gentoo. Its use is as follows for linux/arm64 :
1#Dockerfile.arm64
2# We need to set the platform as the build platform since we are cross-compiling.
3FROM gentoo/stage3:musl
4
5RUN emerge --sync
6RUN MAKEOPTS="-j$(nproc)" USE="gold" emerge sys-devel/binutils dev-vcs/git
7RUN MAKEOPTS="-j$(nproc)" \
8 ACCEPT_KEYWORDS="~*" \
9 emerge ">=dev-lang/go-1.21.0"
10
11# Setup crossdev
12RUN MAKEOPTS="-j$(nproc)" emerge sys-devel/crossdev
13RUN mkdir -p /var/db/repos/crossdev/{profiles,metadata} \
14 && echo 'crossdev' > /var/db/repos/crossdev/profiles/repo_name \
15 && echo 'masters = gentoo' > /var/db/repos/crossdev/metadata/layout.conf \
16 && chown -R portage:portage /var/db/repos/crossdev \
17 && printf '[crossdev]\nlocation = /var/db/repos/crossdev\npriority = 10\nmasters = gentoo\nauto-sync = no' > /etc/portage/repos.conf \
18 && crossdev --target aarch64-unknown-linux-musl
19
20# TODO: install dependencies with `static-libs` for aarch64
We don't need to compile Go for other platforms, as it's already a cross-compiler. However, dependencies must be statically compiled in aarch64 :
1# ...
2
3RUN MAKEOPTS="-j$(nproc)" \
4 ACCEPT_KEYWORDS="~*" \
5 CPU_FLAGS_ARM="v8 vfpv3 neon vfp" \
6 USE="static-libs" \
7 aarch64-unknown-linux-musl-emerge --newuse \
8 net-misc/curl \
9 dev-libs/openssl \
10 net-libs/nghttp2 \
11 sys-libs/zlib \
12 net-dns/c-ares
Then, we set the target platform at the compilation step:
1# ...
2
3COPY main.go .
4
5RUN GOARCH=arm64 \
6 CGO_ENABLED=1 \
7 CC="aarch64-unknown-linux-musl-gcc" \
8 CXX="aarch64-unknown-linux-musl-g++" \
9 PKG_CONFIG="aarch64-unknown-linux-musl-pkg-config" \
10 go build -ldflags '-extldflags "-static"' -o main main.go
11
Finally, we can run it:
1docker build -t builder -f Dockerfile.arm64 .
Let's check if it compiled for arm64:
1mkdir -p out
2docker run --rm \
3 -v $(pwd)/out:/out \
4 builder \
5 cp /main /out/main
6
7file ./out/main
8# ELF 64-bit LSB executable, ARM aarch64, statically linked
Pretty good, huh?
Containerization and multi-arch manifests 🔗
Remember that a static binary can run on distroless? Let's build a multi-arch container manifest with our static binaries.
First step is to create a multi-arch cross-compilation environment. Earlier, we did a Dockerfile.arm64 which only targets arm64 from an linux/amd64 platform. Let's make one Dockerfile that targets linux/amd64, linux/arm64, linux/riscv64 and windows/amd64 and that supports any linux build platform.
I'm going to use Podman to build my image, you can also use Docker Buildx:
1#Dockerfile.multi
2FROM --platform=${BUILDPLATFORM} gentoo/stage3:musl AS builder
3
4RUN emerge --sync
5RUN MAKEOPTS="-j$(nproc)" USE="gold" emerge sys-devel/binutils dev-vcs/git
6RUN MAKEOPTS="-j$(nproc)" \
7 ACCEPT_KEYWORDS="~*" \
8 emerge ">=dev-lang/go-1.21.0"
9
10RUN MAKEOPTS="-j$(nproc)" emerge sys-devel/crossdev
11RUN mkdir -p /var/db/repos/crossdev/{profiles,metadata} \
12 && echo 'crossdev' > /var/db/repos/crossdev/profiles/repo_name \
13 && echo 'masters = gentoo' > /var/db/repos/crossdev/metadata/layout.conf \
14 && chown -R portage:portage /var/db/repos/crossdev \
15 && printf '[crossdev]\nlocation = /var/db/repos/crossdev\npriority = 10\nmasters = gentoo\nauto-sync = no' > /etc/portage/repos.conf \
16 && crossdev --target x86_64-unknown-linux-musl \
17 && crossdev --target aarch64-unknown-linux-musl \
18 && crossdev --target riscv64-unknown-linux-musl
19
20RUN MAKEOPTS="-j$(nproc)" \
21 ACCEPT_KEYWORDS="~*" \
22 USE="static-libs" \
23 x86_64-unknown-linux-musl-emerge --newuse \
24 net-misc/curl \
25 dev-libs/openssl \
26 net-libs/nghttp2 \
27 sys-libs/zlib \
28 net-dns/c-ares
29
30RUN MAKEOPTS="-j$(nproc)" \
31 ACCEPT_KEYWORDS="~*" \
32 CPU_FLAGS_ARM="v8 vfpv3 neon vfp" \
33 USE="static-libs" \
34 aarch64-unknown-linux-musl-emerge --newuse \
35 net-misc/curl \
36 dev-libs/openssl \
37 net-libs/nghttp2 \
38 sys-libs/zlib \
39 net-dns/c-ares
40
41RUN MAKEOPTS="-j$(nproc)" \
42 ACCEPT_KEYWORDS="~*" \
43 USE="static-libs" \
44 riscv64-unknown-linux-musl-emerge --newuse \
45 net-misc/curl \
46 dev-libs/openssl \
47 net-libs/nghttp2 \
48 sys-libs/zlib \
49 net-dns/c-ares
50
51COPY main.go .
52
53ARG TARGETOS TARGETARCH
54
55RUN if [ "${TARGETARCH}" = "amd64" ]; then \
56 export CC="x86_64-unknown-linux-musl-gcc"; \
57 export CXX="x86_64-unknown-linux-musl-g++"; \
58 export PKG_CONFIG="x86_64-unknown-linux-musl-pkg-config"; \
59 elif [ "${TARGETARCH}" = "arm64" ]; then \
60 export CC="aarch64-unknown-linux-musl-gcc"; \
61 export CXX="aarch64-unknown-linux-musl-g++"; \
62 export PKG_CONFIG="aarch64-unknown-linux-musl-pkg-config"; \
63 elif [ "${TARGETARCH}" = "riscv64" ]; then \
64 export CC="riscv64-unknown-linux-musl-gcc"; \
65 export CXX="riscv64-unknown-linux-musl-g++"; \
66 export PKG_CONFIG="riscv64-unknown-linux-musl-pkg-config"; \
67 fi; \
68 GOOS=${TARGETOS} \
69 GOARCH=${TARGETARCH} \
70 CGO_ENABLED=1 \
71 go build -ldflags '-extldflags "-static"' -o main main.go
72
73FROM scratch
74
75COPY --from=builder /main .
76
77ENTRYPOINT [ "/main" ]
TARGETPLATFORM, TARGETARCH, TARGETVARIANT, TARGETOS, BUILDPLATFORM, BUILDARCH, BUILDOS and BUILDVARIANT are automatically filled by Docker Buildx or Podman.
Let's build:
1podman build \
2 --manifest builder:multi \
3 --platform linux/amd64,linux/arm64/v8,linux/riscv64 \
4 -f Dockerfile.multi .
5# OR
6docker buildx \
7 --platform linux/amd64,linux/arm64/v8,linux/riscv64 \
8 -t builder:multi \
9 -f Dockerfile.multi .
10# You can also push the image by adding --push.
And that's it! The manifest contains multiple images for each architecture. By pushing the manifest, there will only be one endpoint for multiple architecture:
1podman manifest push --rm --all builder:multi "docker://ghcr.io/darkness4/multi-arch-example:latest"
Splitting the base image 🔗
Sometime, the builder image is too big, and you may not wish to build the base image again. We can split the Dockerfile into two:
1#Dockerfile.base
2FROM gentoo/stage3:musl
3
4RUN emerge --sync
5RUN MAKEOPTS="-j$(nproc)" USE="gold" emerge sys-devel/binutils dev-vcs/git
6RUN MAKEOPTS="-j$(nproc)" \
7 ACCEPT_KEYWORDS="~*" \
8 emerge ">=dev-lang/go-1.21.0"
9
10RUN MAKEOPTS="-j$(nproc)" emerge sys-devel/crossdev
11RUN mkdir -p /var/db/repos/crossdev/{profiles,metadata} \
12 && echo 'crossdev' > /var/db/repos/crossdev/profiles/repo_name \
13 && echo 'masters = gentoo' > /var/db/repos/crossdev/metadata/layout.conf \
14 && chown -R portage:portage /var/db/repos/crossdev \
15 && printf '[crossdev]\nlocation = /var/db/repos/crossdev\npriority = 10\nmasters = gentoo\nauto-sync = no' > /etc/portage/repos.conf \
16 && crossdev --target x86_64-unknown-linux-musl \
17 && crossdev --target aarch64-unknown-linux-musl \
18 && crossdev --target riscv64-unknown-linux-musl
19
20RUN MAKEOPTS="-j$(nproc)" \
21 ACCEPT_KEYWORDS="~*" \
22 USE="static-libs" \
23 x86_64-unknown-linux-musl-emerge --newuse \
24 net-misc/curl \
25 dev-libs/openssl \
26 net-libs/nghttp2 \
27 sys-libs/zlib \
28 net-dns/c-ares
29
30RUN MAKEOPTS="-j$(nproc)" \
31 ACCEPT_KEYWORDS="~*" \
32 CPU_FLAGS_ARM="v8 vfpv3 neon vfp" \
33 USE="static-libs" \
34 aarch64-unknown-linux-musl-emerge --newuse \
35 net-misc/curl \
36 dev-libs/openssl \
37 net-libs/nghttp2 \
38 sys-libs/zlib \
39 net-dns/c-ares
40
41RUN MAKEOPTS="-j$(nproc)" \
42 ACCEPT_KEYWORDS="~*" \
43 USE="static-libs" \
44 riscv64-unknown-linux-musl-emerge --newuse \
45 net-misc/curl \
46 dev-libs/openssl \
47 net-libs/nghttp2 \
48 sys-libs/zlib \
49 net-dns/c-ares
50
Which you can build and push for the build platform:
1docker build -t builder:base-amd64 -f Dockerfile.base
2docker push builder:base-amd64
Then use it in the second Dockerfile:
1FROM --platform=${BUILDPLATFORM} builder:base-amd64 AS builder
2
3COPY main.go .
4
5ARG TARGETOS TARGETARCH
6
7RUN if [ "${TARGETARCH}" = "amd64" ]; then \
8 export CC="x86_64-unknown-linux-musl-gcc"; \
9 export CXX="x86_64-unknown-linux-musl-g++"; \
10 export PKG_CONFIG="x86_64-unknown-linux-musl-pkg-config"; \
11 elif [ "${TARGETARCH}" = "arm64" ]; then \
12 export CC="aarch64-unknown-linux-musl-gcc"; \
13 export CXX="aarch64-unknown-linux-musl-g++"; \
14 export PKG_CONFIG="aarch64-unknown-linux-musl-pkg-config"; \
15 elif [ "${TARGETARCH}" = "riscv64" ]; then \
16 export CC="riscv64-unknown-linux-musl-gcc"; \
17 export CXX="riscv64-unknown-linux-musl-g++"; \
18 export PKG_CONFIG="riscv64-unknown-linux-musl-pkg-config"; \
19 fi; \
20 GOOS=${TARGETOS} \
21 GOARCH=${TARGETARCH} \
22 CGO_ENABLED=1 \
23 go build -ldflags '-extldflags "-static"' -o main main.go
24
25FROM scratch
26
27COPY --from=builder /main .
28
29ENTRYPOINT [ "/main" ]
1podman build \
2 --manifest builder:multi \
3 --platform linux/amd64,linux/arm64/v8,linux/riscv64 \
4 -f Dockerfile.multi .
5# OR
6docker buildx \
7 --platform linux/amd64,linux/arm64/v8,linux/riscv64 \
8 -t builder:multi \
9 -f Dockerfile.multi .
If you wish to have a multi-platform base image (which I do not recommend), you will have to use qemu-user-static.
Conclusion 🔗
This is how I compiled fc2-live-dl-go. I created several base images with Portage and Crossdev, and used the base images to compile my project.
For OS X, I had to set up my own cross-compiling environment with OSXCross. There's no build system to help me, so I had to download and compile the dependencies manually.
The great thing about splitting the base image is that you can now run the Go compilation step in a CI:
1build-export-static:
2 name: Build and export static Docker
3 runs-on: ubuntu-latest
4
5 steps:
6 - uses: actions/checkout@v4
7
8 - name: Set up QEMU
9 uses: docker/setup-qemu-action@v3
10
11 - name: Set up Docker Context for Buildx
12 run: |
13 docker context create builders
14
15 - name: Set up Docker Buildx
16 id: buildx
17 uses: docker/setup-buildx-action@v3
18 with:
19 version: latest
20 endpoint: builders
21
22 - name: Login to GitHub Container Registry
23 uses: docker/login-action@v3
24 with:
25 registry: ghcr.io
26 username: ${{ github.actor }}
27 password: ${{ secrets.GITHUB_TOKEN }}
28
29 - name: Get the oci compatible version
30 if: startsWith(github.ref, 'refs/tags')
31 id: get_version
32 run: |
33 echo "VERSION=$(echo ${GITHUB_REF#refs/*/})" >> $GITHUB_OUTPUT
34 echo "OCI_VERSION=$(echo ${GITHUB_REF#refs/*/} | sed 's/+/-/g' | sed -E 's/v(.*)/\1/g' )" >> $GITHUB_OUTPUT
35
36 - name: Build and export dev
37 uses: docker/build-push-action@v5
38 with:
39 file: Dockerfile.static
40 platforms: linux/amd64,linux/arm64,linux/riscv64
41 push: true
42 tags: |
43 ghcr.io/example/example:dev
44 cache-from: type=gha
45 cache-to: type=gha,mode=max
46
47 - name: Build and export
48 if: startsWith(github.ref, 'refs/tags')
49 uses: docker/build-push-action@v5
50 with:
51 file: Dockerfile.static
52 platforms: linux/amd64,linux/arm64,linux/riscv64
53 push: true
54 tags: |
55 ghcr.io/example/example:latest
56 ghcr.io/example/example:${{ steps.get_version.outputs.OCI_VERSION }}
57 cache-from: type=gha
58 cache-to: type=gha,mode=max