The Triton project from OpenAI is at the forefront of a groundbreaking movement to democratize AI accelerators and GPU kernel programming. It provides a powerful and flexible framework for writing high performance GPU kernels. As AI workloads become increasingly complex, developers need efficient, scalable and reproducible development environments to build and optimize models quickly.
A container-first approach enhances the Triton development workflow by simplifying setup, driving greater consistency and enabling more seamless collaboration. Read on to learn about the benefits of containerized Triton development and how Red Hat’s triton-dev-containers streamline the process.
Red Hat’s Emerging Technologies blog includes posts that discuss technologies that are under active development in upstream open source communities and at Red Hat. We believe in sharing early and often the things we’re working on, but we want to note that unless otherwise stated the technologies and how-tos shared here aren’t part of supported products, nor promised to be in the future.
What is container-first development?
A container-first approach1 means prioritizing containerized environments—like Docker and Podman-throughout the development lifecycle. Containers encapsulate applications, dependencies, configurations and runtime environments into a lightweight, portable unit. For Triton, where performance consistency and ease of deployment are crucial, this approach can help provide:
- Reproducible builds across different environments
- Minimal setup time helps eliminate dependency conflicts
- Predictable execution with optimized resource management
Why container-first development matters?
AI development is moving at an unprecedented pace, requiring cutting edge tools and workflows to keep up. Containerization offers a solution by standardizing environments, reducing setup friction and improving security. Container first strategy is ideal for Triton development because of its:
- Consistent environments: Users can avoid situations where code functions correctly in one environment but fails in another.
- Faster iteration: Users can quickly spin up preconfigured environments for testing and development.
- Improved collaboration: Standardized tools and dependencies make it easy for teams to work together.
Challenges in Triton development without containers
Currently, Triton developers manually set up their environments by cloning the GitHub repository and installing dependencies. This approach introduces several challenges, such as:
- Time intensive setup: Manually configuring development environments slows productivity
- System configuration issues: Differences in local environments cause inconsistencies and subtle bugs
- Incomplete documentation: Unclear setup instructions can result in troubleshooting headaches
- Dependency conflicts: Version mismatches and software conflicts disrupt workflow efficiency
Simplifying Triton development with Red Hat’s dev containers
To address these challenges, Red Hat has created triton-dev-containers2, a repository providing pre-configured containerized environments for Triton development.
The repository includes two types of containers, each designed to support different development workflows:
- Vanilla containers: These serve as base containers where developers can mount their own development directory, offering flexibility for customized setups.
- VS code DevContainers: Designed specifically for Visual Studio Code, these containers integrate seamlessly with the Dev Containers extension.
These development containers are built to support Triton on common GPUs and Triton-CPU for non-GPU workloads. Additionally, All container images are based on Red Hat Enterprise Linux (RHEL) Universal Base Image (UBI)3, to help provide: enterprise grade reliability and security capabilities, redistributable container images and support for both root and non-root users, aligning with best practices for multiuser environments and cloud native deployments. These containers work seamlessly with Docker and Podman, providing flexibility across different container runtimes.
How to get started with the vanilla triton-dev-containers
Assuming you have the following prerequisites:
- git
- Docker or Podman
- Appropriate toolkit for the associated GPU.
- Visual Studio Code Dev Container Extension (only needed for Visual Studio Code dev containers)
Clone the repo:
$ git clone https://github.com/redhat-et/triton-dev-containers.git
Build the images
$ make all OR $ make triton-image
Run one of the development containers:
$ make triton-run
Triton checks inside the containers
The development containers come with a GPU/Triton/Pytorch check script ‘/workspace/triton-gpu-check.py’ that can be run at startup time to ensure GPU access. Below is an example run:
(workspace) python /workspace/triton-gpu-check.py -------------------------------------------------- System Check: PyTorch and Triton -------------------------------------------------- -------------------------------------------------- PyTorch Information -------------------------------------------------- Pytorch version : 2.5.1+rocm6.2 Pytorch detected GPU? : True Pytorch detected GPU name : AMD Instinct MI210 Pytorch HIP version : 6.2.41133-dd7f95766 -------------------------------------------------- Triton Information -------------------------------------------------- Triton version : 3.1.0 Triton backend : hip -------------------------------------------------- Tensor Tests -------------------------------------------------- CPU Tensor: tensor([1., 2., 3., 4.]) is_cuda: False GPU Tensor: tensor([1., 2., 3., 4.], device='cuda:0') is_cuda: True --------------------------------------------------
How to get started with the Triton Visual Studio Code devcontainers4
To get started, confirm all necessary dependencies are installed as per the Visual Studio Code Dev Containers documentation5. Then, either copy the .devcontainer directory to your Triton directory or use the following steps (depending on your hardware) to clone the necessary files within your Triton repository directory.
For CUDA Support
git clone -n --depth=1 --filter=tree:0 https://github.com/redhat-et/triton-dev-containers.git .devcontainer cd .devcontainer git sparse-checkout set --no-cone .devcontainer/setup.sh .devcontainer/triton git checkout mv .devcontainer/triton ./ mv .devcontainer/setup.sh ./ rm -rf .devcontainer/ .git
For ROCm support
git clone -n --depth=1 --filter=tree:0 https://github.com/redhat-et/triton-dev-containers.git .devcontainer cd .devcontainer git sparse-checkout set --no-cone .devcontainer/setup.sh .devcontainer/triton-amd git checkout mv .devcontainer/triton-amd ./ mv .devcontainer/setup.sh ./ rm -rf .devcontainer/ .git
For CPU support
git clone -n --depth=1 --filter=tree:0 https://github.com/redhat-et/triton-dev-containers.git .devcontainer cd .devcontainer git sparse-checkout set --no-cone .devcontainer/setup.sh .devcontainer/triton-cpu git checkout mv .devcontainer/triton-cpu ./ mv .devcontainer/setup.sh ./ rm -rf .devcontainer/ .git
Run the setup script
After setting up the container files, run the following commands:
echo "export USER_UID=`id -u`" >> "${HOME}/.bashrc";
echo "export USER_GID=`id -g`" >> "${HOME}/.bashrc";
source "${HOME}/.bashrc"
./setup.sh
Running the DevContainer in Visual Studio Code
Bring up the Visual Studio Code Command Pallette and search for Dev Containers: Reopen in Container. Visual Studio Code will reload inside the container, providing an optimized development environment.
Why do the containers install some dependencies at startup time?
To optimize image size, certain dependencies are installed at runtime instead of being bundled in the base image. This provides lightweight containers without unnecessary bloat, faster downloads and deployments and flexibility to install only what’s needed.To see these containers in action, check out the Triton development container demo and the Triton development container in Visual Studio Code demo.
By leveraging Red Hat’s Triton development containers, developers get a streamlined, high quality environment for building and optimizing machine learning models without the need to configure dependencies from scratch.
As Triton adoption grows, embracing containerization will ensure efficient workflows, improved collaboration and production ready AI models. Red Hat’s Triton Dev Containers provide a powerful, reliable, and developer friendly way to build AI workloads—without the traditional software development pain points.Ready to get started? Check out the repository here!
- Docker. “5 Benefits of a Container-First Approach to Software Development.” Docker Blog, 7 Feb. 2023, https://www.docker.com/blog/5-benefits-of-a-container-first-approach-to-software-development/. Accessed 12 Feb. 2025. ↩︎
- Red Hat. “Triton Dev Containers.” GitHub, https://github.com/redhat-et/triton-dev-containers. Accessed 12 Feb. 2025. ↩︎
- Red Hat. Introducing Red Hat Universal Base Image. Red Hat, https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image. Accessed 12 Feb. 2025. ↩︎
- Visual Studio Code. Developing Inside a Container. Microsoft, https://code.visualstudio.com/docs/devcontainers/containers. Accessed 12 Feb. 2025. ↩︎
- Visual Studio Code. “Prerequisites – Developing Inside a Container.” Microsoft, https://code.visualstudio.com/docs/devcontainers/tutorial#_prerequisites. Accessed 12 Feb. 2025. ↩︎
