dockcross: C++ Write Once, Run Anywhere

Triangle C++ Developers Group

November 17, 2016

Matt McCormick, PhD

Kitware, Inc

https://github.com/dockcross/cxx-write-once-run-anywhere

Motivation

"Write Once, Run Anywhere"

Java Logo

Claim by Java programmers.

Avoid Controversy

EMACS SUX!

VIM RULEZ!

vim rulez

FactCheck: Java runs everywhere?

The Java JVM

You didn't fact check that!

  • If you have a JVM installed...

FactCheck: C++ runs everywhere

  • C/C++ compilers are available for more platforms than any other language ✔
  • We can effectively run C++ code on any platform ?

Not true

What do we need to make it happen?

  • Good build system support for cross-compilation
  • Easy access to ready-to-use cross-compilation toolchains

dockcross

https://github.com/dockcross/dockcross

  • Good build system support for cross-compilation → CMake
  • Easy access to ready-to-use cross-compilation toolchains → Docker images

Open Source on GitHub

dockcross GitHub

https://github.com/dockcross/dockcross

CircleCI Deployed Images Freely Available on DockerHub

dockcross DockerHub

https://hub.docker.com/u/dockcross

Available Cross Compilers

  • dockcross/base: Base image for other toolchain images. From Debian Jessie with GCC, make, autotools, CMake, Ninja, Git, and Python.
  • dockcross/android-arm: The Android NDK standalone toolchain for the arm architecture.
  • dockcross/browser-asmjs: The Emscripten JavaScript cross compiler.

Available Cross Compilers (cont.)

  • dockcross/linux-arm64: Cross compiler for the 64-bit ARM platform on Linux, also known as AArch64.
  • dockcross/linux-armv5: Linux armv5 cross compiler toolchain for legacy devices like the Parrot AR Drone.
  • dockcross/linux-armv6: Linux ARMv6 cross compiler toolchain for the Raspberry Pi, etc.
  • dockcross/linux-armv7: Generic Linux armv7 cross compiler toolchain.

Available Cross Compilers (cont.)

  • dockcross/linux-ppc64le: Linux PowerPC 64 little endian cross compiler toolchain for the POWER8, etc.
  • dockcross/linux-x64: Linux x86_64 / amd64 compiler. Since the Docker image is natively x86_64, this is not actually a cross compiler.
  • dockcross/linux-x86: Linux i686 cross compiler.

Available Cross Compilers (cont.)

  • dockcross/manylinux-x64: Docker manylinux image for building Linux x86_64 / amd64 Python wheel packages.
  • dockcross/manylinux-x86: Docker manylinux image for building Linux i686 Python wheel packages.

Available Cross Compilers (cont.)

  • dockcross/windows-x64: 64-bit Windows cross-compiler based on MXE/MinGW-w64.
  • dockcross/windows-x86: 32-bit Windows cross-compiler based on MXE/MinGW-w64.

Not Available: Mac OSX

A cross-compiling toolchain exists, but Apple's SDK terms of service states:

You agree not to rent, lease, lend, upload to or host on any website or server, sell, redistribute, or sublicense the Apple Software and Apple Services, in whole or in part, or to enable others to do so.

and

You are expressly prohibited from separately using the Apple SDKs or attempting to run any part of the Apple Software on non-Apple-branded hardware.

Cross Platform

Linux Mac Windows CircleCI

Example

In [2]:
cd example
find .
.
./src
./src/IntrospectionDemo.cxx
./src/IntrospectionDemoConfigure.h.in
./src/CMakeLists.txt
./build

Get the dockcross Script

In [4]:
docker pull dockcross/windows-x64
docker run --rm dockcross/windows-x64 > dockcross-windows-x64
chmod +x ./dockcross-windows-x64
Using default tag: latest
latest: Pulling from dockcross/windows-x64



















Digest: sha256:be93869c5df9c17cc1fbc430bb3ed75d73ba0c5d5a6704f4811792659f7a0878
Status: Downloaded newer image for dockcross/windows-x64:latest

Build the Project

In [5]:
./dockcross-windows-x64 cmake -Hsrc -Bbuild -GNinja
./dockcross-windows-x64 ninja -Cbuild
-- The C compiler identification is GNU 4.9.4
-- The CXX compiler identification is GNU 4.9.4
-- Check for working C compiler: /usr/src/mxe/usr/bin/x86_64-w64-mingw32.static-gcc
-- Check for working C compiler: /usr/src/mxe/usr/bin/x86_64-w64-mingw32.static-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/src/mxe/usr/bin/x86_64-w64-mingw32.static-g++
-- Check for working CXX compiler: /usr/src/mxe/usr/bin/x86_64-w64-mingw32.static-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of long
-- Check size of long - done
-- Double correction test returned: 89255.0/1e22 = 8.9255e-018
-- Configuring done
-- Generating done
-- Build files have been written to: /work/build
ninja: Entering directory `build'
[2/2] Linking CXX executable IntrospectionDemo.exe

Problems Solved

Toolchain Configuration

  • Pre-built and configured toolchains.
  • Most images also contain an emulator for the target system.
  • Make variables (CC, LD etc) are set to point to the appropriate tools in the container.
  • Recent CMake and ninja are precompiled.
  • Toolchain files configured for CMake.

Frustrated Jackie Chan

Portable, Reproducible Build Environment

  • Download the build system with a simple docker pull
  • Commands in the container are run as the calling user, so that any created files have the expected ownership, (i.e. not root).
  • Clean build environment separation -- every run is a new Docker container

Compiler-based Platform Introspection

CMake has a try_compile command to perform compiler-based platform introspection. It learns about the system by trying to compile a small source file.

# The CheckTypeSize CMake module uses the
# try_compile command internally
include(CheckTypeSize)
check_type_size("long" IntrospectionDemo_SIZEOF_LONG)

configure_file(IntrospectionDemoConfigure.h.in
  IntrospectionDemoConfigure.h)
In [6]:
rm -rf build/*
./dockcross-windows-x86 cmake -Hsrc -Bbuild -GNinja | tail -n 6
-- Check size of long
-- Check size of long - done
-- Double correction test returned: 89255.0/1e22 = 8.9255e-018
-- Configuring done
-- Generating done
-- Build files have been written to: /work/build
In [7]:
grep SIZEOF_LONG ./build/IntrospectionDemoConfigure.h
#define IntrospectionDemo_SIZEOF_LONG 4
In [8]:
rm -rf build/*
./dockcross-linux-arm64 cmake -Hsrc -Bbuild -GNinja | tail -n 6




In [9]:
grep SIZEOF_LONG ./build/IntrospectionDemoConfigure.h
#define IntrospectionDemo_SIZEOF_LONG 8

Runtime-based Platform Introspection

CMake also has a try_run command to perform command execution-based platform introspection. It learns about the system by trying to run a small source file. An emulator can be used to automatically generate this information when cross-compiling.

To determine if there is an 80 bit or 64 bit floating point stack

set(_test_source "${CMAKE_CURRENT_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/double-correction-needed.cc")
file(WRITE ${_test_source} "
#include <stdio.h>
int main(int, char **)
{
  const double correct = 89255.0/1e22;
  printf(\"89255.0/1e22 = %g\", correct);
  if( correct != 89255e-22 )
    {
    // correction required
    return 0;
    }
  return 1;
}")
try_run(IntrospectionDemo_DOUBLE_CORRECTION_NEEDED
  DOUBLE_CORRECT_NEEDED_COMPILED
  ${CMAKE_CURRENT_BINARY_DIR} ${_test_source}
  RUN_OUTPUT_VARIABLE _double_correction_out)
message(STATUS "Double correction test returned: ${_double_correction_out}")
In [10]:
rm -rf build/*
./dockcross-linux-arm64 cmake -Hsrc -Bbuild -GNinja | tail -n 4


Test Execution

How do we know the build works -- we need to test it!

Tests can be executed with the target system emulator.

In [11]:
./dockcross-linux-arm64 ninja -Cbuild
ninja: Entering directory `build'
[2/2] Linking CXX executable IntrospectionDemo
In [12]:
./dockcross-linux-arm64 bash -c 'cd build && ctest -V'
UpdateCTestConfiguration  from :/work/build/DartConfiguration.tcl
Parse Config file:/work/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/work/build/DartConfiguration.tcl
Parse Config file:/work/build/DartConfiguration.tcl
Test project /work/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: IntrospectionDemoTest

1: Test command: /usr/bin/qemu-aarch64 "/work/build/IntrospectionDemo"
1: Test timeout computed to be: 1500
1: 
1: The 'long' type has 8 bytes.
1: This platform needs double correction.
1: 
1/1 Test #1: IntrospectionDemoTest ............   Passed    0.04 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.04 sec

Check It Out!

  • https://github.com/dockcross/dockcross
  • Open source collaboration welcome

GitHub - https://github.com/thewtex

https://github.com/thewtex

Twitter - https://twitter.com/thewtex

https://twitter.com/thewtex

Kitware - http://www.kitware.com/

http://www.kitware.com