Building VTK

Building VTK#

One command builds a VTK wheel from source:

pyvista-build vtk

It works on Linux, macOS, and Windows and follows the official Building Python wheels instructions, with the options PyVista’s own CI uses. A first build takes from minutes to over an hour and needs several GB of disk.

  • The wheel is built for, and installed into, the active virtual environment, else .venv in the current directory, else the Python running the command. --python PATH picks another interpreter.

  • It says where the build goes and asks once before starting, then builds with those defaults straight through. Anything missing that it would install is listed before that question, which covers it too. --yes answers it in advance.

  • Every command is printed as it runs. Its output shows as one line that updates as it runs, with the time elapsed. --verbose (-v) prints all of the output and the environment variables each command sets.

--interactive walks through the build one step at a time:

  • Each step shows its settings, says what it is about to do and where, and waits for a yes.

  • Enter accepts the default, n goes through the settings one by one, and the same flags the command line takes change one in place.

pyvista-build vtk --interactive

pyvista-build --help lists every flag. The result is a wheel, so once installed (manually or with --install) PyVista picks it up like the stock one.

The Steps#

Every build runs the same eight steps, in this order. Each one prints what it is about to do, and --interactive waits for a yes before it runs. See the dry run for sample output for each step.

1. Tools#

Ensure the tools the build needs are available.

  • Checks git, cmake, ninja, a C++ compiler, the wheel repair tool, and patchelf on Linux before anything is downloaded. See Tooling for what each one does.

  • Lists everything the build would install, each item with where it goes: cmake, ninja, the repair tool, and patchelf on Linux when missing or too old, setuptools and wheel, and the finished wheel with --install.

  • Starting the build covers that whole list. --interactive asks again at this step, and --yes answers in advance. Without a terminal and without --yes the command stops instead of installing.

  • The compiler has to compile and link a one-line program. A missing or broken one stops with instructions for your operating system, such as an Xcode and Command Line Tools pair that no longer match after a macOS update. On Windows, Visual Studio’s build environment is imported automatically.

  • A compiler cache (ccache or sccache) is used when one is on PATH. --compiler-cache none builds without one.

  • When the cloned source asks for other cmake or ninja versions than the ones checked, as on a first cvista build, those are checked again after step 2.

2. Source#

Get the VTK source to build.

  • Clones VTK into ./vtk-build/src at the release tag PyVista supports.

  • --ref picks another tag, branch, or commit. --build-dir puts everything somewhere else. --source DIR builds a checkout you already have.

  • An existing checkout is reused as is, local edits included.

  • With --interactive the clone prompt also takes a fork’s git URL or the path of another checkout, and an existing checkout is confirmed before it is reused. Answering with a git URL there deletes it, after one more confirmation, and clones the fork in its place.

  • A build tree configured from one checkout cannot build another, so switching checkouts asks before deleting the build tree. --clean or another --build-dir avoids the question.

3. Build options#

Choose which modules and options to build.

On top of the modules every build has, the modules to build are organized into named groups. The default set matches the stock wheel plus what PyVista’s CI enables:

  • rendering: OpenGL rendering, widgets, charts, and text

  • io: extra readers and writers (scene import, Exodus, Xdmf2)

  • web: web export modules used by trame and the Jupyter backends

  • matplotlib: Matplotlib-rendered math text and labels

  • parallel: DIY2-based parallel filters

  • serialization: object serialization and .pyi stub files

  • headless (Linux, off by default): no X, rendering through EGL or OSMesa, see Off-Screen Plotting GPU Support

Options:

  • --with and --without add or drop groups, --minimal keeps only the required ones.

  • --opt smp=VALUE picks the threading backend for SMP-enabled filters: Sequential (the default, as in the PyPI wheels), STDThread, OpenMP, or TBB, which needs TBB installed.

  • At the prompt, n asks about each group in turn, and the command-line flags work there as well.

pyvista-build vtk --without web,matplotlib
pyvista-build vtk --minimal --with rendering
pyvista-build vtk --opt smp=STDThread

4. Configure#

Generate the build tree with CMake.

  • Shows the build type, compiler cache, version suffix, and extra CMake cache entries, then runs cmake.

  • --build-type, --compiler-cache, --version-suffix, and -D change one directly. At the prompt, n asks about each in turn.

  • Every command is printed in full before it runs.

5. Build#

Compile VTK.

  • Runs cmake --build with Ninja and logs the full output to vtk-build/build.log.

  • Uses one compile job per CPU core, at most one per 2 GB of memory, since each job can take that much. --jobs sets the number instead.

  • When a command fails, its last lines of output are printed and the build stops. The log has the complete compiler output.

6. Wheel#

Package the build as a wheel.

  • Runs setup.py bdist_wheel in the build tree.

7. Repair#

Make the wheel installable outside the build tree.

  • Bundles the libraries the wheel links against and sets its platform tag, so it installs outside the build tree and on other machines.

  • A manifest’s [repair] table can ask for the bundled libraries to be stripped and pass extra arguments to the repair tool.

  • A rerun reuses the repaired wheel when the built wheel’s contents have not changed.

  • --no-repair skips the step.

8. Install#

Install the wheel into the target environment.

  • --install installs the wheel into the target environment, which starting the build covers. Without it, --interactive offers to, and otherwise the wheel’s path and the pip command are printed.

  • The wheel replaces the installed vtk. pip install --force-reinstall vtk goes back to the stock one.

Scripted Use#

Without --interactive the command asks once before it starts, and that answer covers installing anything that is missing. Without a terminal it stops at that question, so pass --yes in CI and scripts, and --verbose so the job log shows every command in full and every line of output. --dry-run prints the commands, and the install list, without running anything. For a backend whose manifest lives only in its source tree, such as cvista, --dry-run needs an existing checkout and stops without one. Optional steps keep their own switches, so --no-repair skips the repair step and --install installs the wheel at the end.

pyvista-build vtk --ref master --build-type RelWithDebInfo --jobs 8 --yes
pyvista-build vtk --dry-run

A backend that always builds the same way in CI keeps those flags in a [defaults.<name>] table of its manifest, and the command line names the set. A backend whose ci set holds --yes, --source . and its build options runs its job as:

pyvista-build cvista --defaults ci

Flags given after the name still win, so --defaults ci --jobs 4 builds the set with four jobs. Manifest Reference describes the table.

Developer Loop#

The build tree is kept between runs. Edit files under vtk-build/src and run pyvista-build vtk again: configure is skipped unless the options changed, ninja rebuilds what the edit touched and a new wheel is written. --reconfigure forces cmake to run again, --clean deletes the build tree first, and -D KEY=VALUE passes any CMake cache entry straight through:

pyvista-build vtk --build-type Debug -D VTK_ENABLE_LOGGING=OFF

--ref leaves a checkout that is already at that commit as it is, edits included, and otherwise checks the commit out, which needs a checkout without local edits. --update fetches first, then checks out --ref, or the release tag when --ref is not given, and always needs a checkout without local edits:

pyvista-build vtk --update --ref master

CI Workflows#

A CI job builds from scratch every time, so a compiler cache is what keeps it short: a warm cache turns a build of hours into minutes.

  • Pass --yes and --verbose, as in Scripted Use.

  • pip install pyvista-build[tools] installs every tool the build gets from PyPI up front, so the tools step has nothing left to install.

  • The build uses ccache or sccache, whichever is on PATH. Install ccache on Linux and macOS (apt-get install ccache, brew install ccache) and set up sccache on Windows, where MSVC needs it (mozilla-actions/sccache-action).

  • --cache-dir puts the cache in a directory the job can save and restore between runs. Save it at the end of the job and restore it at the start, one cache per operating system: object files from one do not help another.

  • The cache tools read their own environment variables, and the command passes every CCACHE_* and SCCACHE_* setting through to the build and into the manylinux container except the host paths: CCACHE_DIR becomes the mounted cache and CCACHE_BASEDIR, CCACHE_CONFIGPATH, CCACHE_LOGFILE and SCCACHE_DIR stay on the host. FORCE_COLOR, NO_COLOR, TERM and COLUMNS go in as well, so the container’s output is colored and wrapped like the host’s, and so does every variable the manifest’s [manylinux] env names. Set FORCE_COLOR=1 in a job to keep the colors in its log. CCACHE_MAXSIZE=1G or SCCACHE_CACHE_SIZE=1G is the one worth setting in a job: the defaults are 5 GB and 10 GB, a full VTK build fits in 1 GB, and the cache is uploaded and downloaded on every run. A developer machine needs no cap.

  • The command already keys ccache entries on the compiler’s version string rather than its binary’s timestamp (CCACHE_COMPILERCHECK) and on paths relative to the build (CCACHE_BASEDIR), and leaves the __DATE__ and __TIME__ macros, the locale variables and the defines of precompiled headers out of the hash (CCACHE_SLOPPINESS), so a runner image update that ships the same compiler, or a job that checks out elsewhere, keeps the cache valid. Each variable set in the environment beforehand wins over these defaults.

  • On Linux add --manylinux for a wheel that installs on other machines; the container installs ccache itself and uses the same directory.

  • After the build, one line says how many of this run’s compilations hit and missed the cache, followed by the cumulative statistics. On GitHub Actions both go to the job summary (GITHUB_STEP_SUMMARY), from inside the container as well.

pyvista-build vtk --yes --verbose --cache-dir compiler-cache

This repository’s own workflow does this on all three operating systems, keeping the cache as a workflow artifact. A full hit rate finishes in about 10 minutes on Linux, 25 on macOS, and 35 on Windows.

Other Backends#

The same command builds any VTK backend whose source tree carries a pyvista-build.toml manifest. That file declares the groups and options the backend offers, its tool requirements, per-platform CMake init caches, and the extra commands it needs; stock VTK’s manifest ships with pyvista-build. cvista, the community fork, is known by name. A fork of VTK is built as vtk from its URL, using the manifest in its tree when it has one and the shipped one otherwise. A --url the existing checkout in the build directory was not cloned from stops the build, so give a fork its own --build-dir:

pyvista-build cvista
pyvista-build vtk --url https://github.com/me/vtk.git --ref my-branch --build-dir vtk-fork-build

Manifest Reference lists every key a manifest can set, for a fork that wants its own groups, options or hooks.

A backend built this way is selected like any other installed one: automatically for cvista, or explicitly through PYVISTA_VTK_BACKEND.

Off-Screen Plotting GPU Support#

The stock Linux wheels already render off-screen without an X server: VTK picks EGL for GPU rendering, or OSMesa for software rendering, at run time. To build a wheel that does not link against X at all:

pyvista-build vtk --with headless

Such a wheel cannot open on-screen windows, so use it only where you save static images or stream the render window to another computer with a display (for example, using pyvista.set_jupyter_backend('server') and JupyterLab).

Portable Linux Wheels#

A wheel built on Linux is tagged for the glibc of the machine that built it and may not install elsewhere. To share one, build it inside a manylinux container, as PyVista’s own CI does.

  • --manylinux builds a Linux wheel only. It runs the build in a container through Docker, so Docker is needed for this flag and for nothing else.

  • Without --manylinux no Docker is involved on any operating system, and macOS and Windows wheels are always built that way.

  • A macOS or Windows machine can still run --manylinux to produce a Linux wheel, which it then cannot install.

Docker must be installed and running first:

  • Linux: install Docker Engine for your distribution, then add yourself to the docker group (sudo usermod -aG docker $USER) and log in again so no sudo is needed.

  • macOS: install Docker Desktop for Mac (or brew install --cask docker) and start it.

  • Windows: install Docker Desktop for Windows with the WSL 2 backend and start it.

docker run --rm hello-world confirms it works. Then:

pyvista-build vtk --manylinux

What happens:

  • The image matches your machine’s architecture (manylinux_2_28_x86_64 or manylinux_2_28_aarch64) and carries the same Python version as the one running the command. --manylinux-image picks another image.

  • The source is cloned or updated on your machine first.

  • The container installs VTK’s build dependencies and this package’s requirements, mounts the pyvista-build you are running read-only, and runs the remaining steps with the source and ./vtk-build mounted.

  • The repair step runs auditwheel there with --plat set to the image’s policy (its AUDITWHEEL_PLAT), which gives the wheel the image’s tag and puts it in vtk-build/wheelhouse.

  • Files written by the container are handed back to your user, even when the build fails.

  • On a Linux host the install step runs afterwards. On macOS and Windows the wheel is only written, since it is for Linux.

Caches and limits:

  • The container installs ccache and keeps its cache between runs in vtk-build/compiler-cache, or in --cache-dir or $CCACHE_DIR when set. --compiler-cache none skips it. sccache is not available there.

  • pip keeps its cache in vtk-build/pip-cache.

  • A build tree configured inside the container cannot be continued outside it.

Tooling#

What the command looks for and offers to install. The tools extra (pip install pyvista-build[tools]) installs every PyPI one up front, here or in CI:

  • git clones the source and checks out the requested tag, branch, or commit. Never installed by the command.

  • CMake configures the build: it reads VTK’s CMakeLists.txt and the options chosen, finds the compiler and Python, and writes the build files. Installed with pip when missing.

  • Ninja is the build tool that CMake generates files for; it runs the compiler across all cores and only rebuilds what changed. Installed with pip when missing.

  • A C++ compiler: GCC or Clang on Linux, Xcode’s command-line tools on macOS (xcode-select --install), Visual Studio with the Desktop development with C++ workload on Windows. Never installed by the command. A missing one stops with the instructions for your operating system.

  • Development headers on Linux for Python, OpenGL, and X11: sudo apt install python3-dev libgl1-mesa-dev libx11-dev libxcursor-dev on Debian and Ubuntu, or sudo dnf install python3-devel mesa-libGL-devel libX11-devel libXcursor-devel on Fedora. The tool check cannot see them, so a missing one stops cmake instead.

  • ccache and sccache are compiler caches: they store each object file keyed by its inputs, so rebuilding after an edit or a reconfigure compiles again only what changed. ccache is the usual choice on Linux and macOS; sccache is the one that supports MSVC on Windows. The build uses whichever is on PATH, and --compiler-cache none turns it off.

  • Wheel repair tools copy the shared libraries a wheel depends on into the wheel itself and set the platform tag, so it installs on other machines: auditwheel on Linux, which also needs patchelf, delocate on macOS, and delvewheel on Windows. Installed with pip when missing.

  • setuptools and wheel are the Python packages that setup.py bdist_wheel needs to package the build tree. Installed with pip when missing.

  • Docker runs the manylinux container for --manylinux. Never installed by the command.

Building by Hand#

pyvista-build vtk --dry-run prints the exact cmake and setup.py commands, which is the quickest way to get a starting point for a manual build. The essential configuration is:

git clone https://gitlab.kitware.com/vtk/vtk.git
cmake -S vtk -B vtk/build -GNinja \
      -DCMAKE_BUILD_TYPE=Release \
      -DVTK_WHEEL_BUILD=ON \
      -DVTK_WRAP_PYTHON=ON \
      -DVTK_BUILD_TESTING=OFF \
      -DVTK_BUILD_DOCUMENTATION=OFF \
      -DVTK_BUILD_EXAMPLES=OFF \
      -DPython3_EXECUTABLE=$(which python)
cmake --build vtk/build --parallel
cd vtk/build && python setup.py bdist_wheel

The wheel is written to vtk/build/dist.

Contents