I needed to wrap a native C library for a cross-platform project in Rust. I decided to use the typical pattern of a *-sys crate and a crate that wraps the sys-crate in a more Rust-like API. This blog post describes the steps I took to wrap the native library in Rust. I decided to write a blog post about it, as all posts I found online where incomplete and left out important details.
As the library I needed to wrap is proprietary, I will use a different library as an example. Let's say the library is called coolmath and it has some interesting algorithms that we want to use in Rust. We do not have any source code for the library. Just a header with the function definitions and the corresponding dynamic (shared) libraries for the different operating systems and architecture are provided. No static libraries are provided, so we need to link at runtime.
We have the following files:
A typical pattern in Rust for wrapping native libraries is to have a *-sys crate that contains the bindings to the native library and a crate that wraps the sys-crate in a more Rust-like API. We will call the sys-crate coolmath-sys and the wrapper crate coolmath. The coolmath-sys crate should be usable as a dependency in other crates, but usually the sys-crate is not used directly. The coolmath crate should be usable as a dependency in other crates and should depend on the coolmath-sys crate.
We start by creating a new Rust crate with cargo new --lib coolmath-sys. In the newly created directory, we create a new directory called vendor. This contains all libraries and headers in sub-folders that we need for our crate. The full directory-tree looks like this:
coolmath-sys
├── Cargo.toml
├── src
│ └── lib.rs
└── vendor
├── inc
│ └── coolmath.h
└── lib
├── libcoolmath_linx64.so
├── libcoolmath_linarm64.so
├── libcoolmath_macx64.dylib
├── libcoolmath_macarm64.dylib
├── coolmath_winx64.dll
└── coolmath_winarm64.dll Next, we need a build.rs file in the root of our crate. This file is used by cargo to build our crate. It runs bindgen to automatically generate the bindings for our native library. Additionally we need a wrapper.h in the root directory of our crate. This header includes all headers from our native library that bindgen needs to generate bindings for. In our case its just one, the coolmath.h, but if you need more headers, add them to the wrapper.h.
// wrapper.h
// Include all headers that bindgen needs to generate bindings for.
#include "vendor/inc/coolmath.h"We need to add bindgen as a dependency to our Cargo.toml file. It has to be a build dependency, as it runs only on build time, to generate the Rust bindings. Add the following code to your Cargo.toml file.
# ... your Cargo.toml
[build-dependencies]
bindgen = "0.69.1"The build.rs is the most interesting part. All the magic to create a sys-crate happens here. You find the build script for our fictional library below.
use std::{env, fs};
use std::path::{PathBuf, Path};
fn main() {
// Tell cargo to look for shared libraries in the specified directory
let libdir_path = PathBuf::from("vendor/lib")
.canonicalize()
.expect("cannot canonicalize vendor path");
println!("cargo:rustc-link-search={}", libdir_path.to_str().unwrap());
println!("cargo:rustc-link-search={}", env::var("OUT_DIR").unwrap());
// If on Linux or MacOS, tell the linker where the shared libraries are
// on runtime (i.e. LD_LIBRARY_PATH)
match target_and_arch() {
(Target::Linux, _) | (Target::MacOS, _) => {
println!("cargo:rustc-link-arg=-Wl,-rpath,{}",
env::var("OUT_DIR").unwrap());
}
_ => {}
}
// Tell cargo to link against the shared library for the specific platform.
// IMPORTANT: On macOS and Linux the shared library must be linked without
// the "lib" prefix and the ".so" suffix. On Windows the ".dll" suffix must
// be omitted.
match target_and_arch() {
(Target::Windows, Arch::X86_64) => {
println!("cargo:rustc-link-lib=coolmath_winx64");
copy_dylib_to_target_dir("coolmath_winx64.dll");
}
(Target::Windows, Arch::AARCH64) => {
println!("cargo:rustc-link-lib=coolmath_winarm64");
copy_dylib_to_target_dir("coolmath_winarm64.dll");
}
(Target::Linux, Arch::X86_64) => {
println!("cargo:rustc-link-lib=coolmath_linx64");
copy_dylib_to_target_dir("libcoolmath_linx64.so");
}
(Target::Linux, Arch::AARCH64) => {
println!("cargo:rustc-link-lib=coolmath_linarm64");
copy_dylib_to_target_dir("libcoolmath_linarm64.so");
}
(Target::MacOS, Arch::X86_64) => {
println!("cargo:rustc-link-lib=coolmath_macx64");
copy_dylib_to_target_dir("libcoolmath_macx64.dylib");
}
(Target::MacOS, Arch::AARCH64) => {
println!("cargo:rustc-link-lib=coolmath_macarm64");
copy_dylib_to_target_dir("libcoolmath_macarm64.dylib");
}
}
// Tell cargo to invalidate the built crate whenever the wrapper changes
println!("cargo:rerun-if-changed=wrapper.h");
// The bindgen::Builder is the main entry point
// to bindgen, and lets you build up options for
// the resulting bindings.
let bindings = bindgen::Builder::default()
// The input header we would like to generate
// bindings for.
.header("wrapper.h")
.clang_arg("-v")
.derive_debug(true)
.derive_default(true)
// included header files changed.
.parse_callbacks(Box::new(bindgen::CargoCallbacks::new()))
// Finish the builder and generate the bindings.
.generate()
// Unwrap the Result and panic on failure.
.expect("Unable to generate bindings");
// Write the bindings to the $OUT_DIR/bindings.rs file.
let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
bindings
.write_to_file(out_path.join("bindings.rs"))
.expect("Couldn't write bindings!");
}
fn copy_dylib_to_target_dir(dylib: &str) {
let out_dir = env::var("OUT_DIR").unwrap();
let src = Path::new("vendor/lib");
let dst = Path::new(&out_dir);
let _ = fs::copy(src.join(dylib), dst.join(dylib));
}
enum Target {
Windows,
Linux,
MacOS
}
enum Arch {
X86_64,
AARCH64,
}
fn target_and_arch() -> (Target, Arch) {
let os = env::var("CARGO_CFG_TARGET_OS").unwrap();
let arch = env::var("CARGO_CFG_TARGET_ARCH").unwrap();
match (os.as_str(), arch.as_str()) {
// Windows targets
("windows", "x86_64") => (Target::Windows, Arch::X86_64),
("windows", "aarch64") => (Target::Windows, Arch::AARCH64),
// Linux targets
("linux", "x86_64") => (Target::Linux, Arch::X86_64),
("linux", "aarch64") => (Target::Linux, Arch::AARCH64),
// MacOS targets
("macos", "x86_64") => (Target::MacOS, Arch::X86_64),
("macos", "aarch64") => (Target::MacOS, Arch::AARCH64),
_ => panic!("Unsupported operating system {} and architecture {}", os, arch),
}
}There are a few interesting things happening here. First, we tell cargo where to find the shared libraries. Then we check which platform we are building for and link against the correct shared library. We also copy the shared library to the target directory, so it can be found at runtime. Finally, we tell bindgen to generate the bindings for our native library. You may wonder, why environment variable are used to determine the operating system and architecture instead of #[cfg(...)] that you may know from other Rust projects. The problem with the #[cfg(...)] approach is that it does not work correctly for cross-platform builds. You run into ugly linker issues, where the linker tries to link against the wrong platform.
// Alternative to environment variables are #[cfg(...)] directives.
// Unfortunately, they do not work correctly for cross-platform builds.
// For example, the following code does not work:
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
Until now, we have not written any Rust code. We only have a build.rs and a wrapper.h file. We want to export the wrapped C code in our sys-crate, so it can be used by other crates. We do this by adding the following code to our lib.rs file. This includes the generated bindings in our crate code and exports them for others to use.
// src/lib.rs
#![allow(non_upper_case_globals)]
#![allow(non_camel_case_types)]
#![allow(non_snake_case)]
include!(concat!(env!("OUT_DIR"), "/bindings.rs")); In theory, we are done. A cargo build should build our crate and generate the bindings. Unfortunately, there are a few special cases that we need to handle. On Linux, there is nothing more to-do. But on the other platforms there is something missing. MacOS is very similar to Linux but it takes security a bit more serious (don't hate me for that snarky comment) so we need to sign the native libraries libcoolmath_macx64.dylib and libcoolmath_macarm64.dylib with a valid developer certificate, else the library cannot be linked against. Unfortunatly the vendor I got the lib from did not sign their library, so I had to do it. The process is called "notarization" on macOS and you can read more here: Notarizing macOS Software before distribution. If you intend your crate to "just-work" on macOS, make sure your libraries are notarized correctly. After that is done, macOS works exactly like Linux.
Windows is a bit more complicated. We only have a *.dll but the linker needs a *.lib. Why do we need a static library to link against a dynamic one? Windows has the concept of import libraries. They are used to link against dynamic libraries. The import library contains the information that is needed to load the dynamic library at runtime and itself is not needed at runtime, only at compile time. The import library is usually created by the compiler when you compile a dynamic library. But we do not have the source code for the library, so we cannot compile it. We need to create the import library ourselves. There are several ways to do that, but I simply used tools that are already installed, if you have the typical Windows developer tools installed which come with any version of Visual Studio.
I did the following steps to create the import library for the coolmath_winx64.dll and coolmath_arm64.dll.
coolmath-sys
├── Cargo.toml
├── wrapper.h
├── build.rs
├── src
│ └── lib.rs
└── vendor
├── inc
│ └── coolmath.h
└── lib
├── libcoolmath_linx64.so
├── libcoolmath_linarm64.so
├── libcoolmath_macx64.dylib
├── libcoolmath_macarm64.dylib
├── coolmath_winx64.dll
├── coolmath_winx64.lib
├── coolmath_winarm64.dll
└── coolmath_winarm64.lib
The coolmath-sys crate is now ready to be used as a dependency in other crates. You can publish it to crates.io if it contains no proprietary code. In my case I published it to a private instance of Kellnr (obviosly).
As we do not want to use the wrapped C code in the coolmath-sys crate directly, but instead provide a more convinient and safe crate, we crate a nother crate: cargo new --lib coolmath. We add the coolmath-sys crate as a dependency in the Cargo.toml.
# ... your Cargo.toml
[dependencies]
coolmath-sys = { version = "0.1.0", registry = "kellnr" }We can now use the coolmath-sys crate in our coolmath crate. Usually you want to get rid of all unsafe code from the sys-crate by wrapping it in safe Rust in the non-sys crate. As wrapped C code feels a bit clumsy to use from Rust, a idiomatic API is added as well. As I want to set the focus on the "build part" of the cross-platform crate and not on the safe code wrapper, I leave that to the reader. If you are done with your wrapper crate, you can publish it to a registry, too. Now, everyone can use the initial C library in a convinient and safe way from Rust. We accomplished our mission.
We created a cross-platform Rust crate that wraps a native C library. We used the typical pattern of a *-sys crate and a crate that wraps the sys-crate in a more Rust-like API. The build.rs file we created, deals with the different shared libraries and linker options for the different platforms we want to support. Pitfals like missing *.lib files or notarization on macOS are handled as well.
I hope you had some fun reading the blog post and it is of use for your next cross-platform Rust project.