Triton Kernel Profiling with NVIDIA Nsight Tools

Are your custom Triton GPU kernels running as efficiently as they could be? Unlocking peak performance requires the right tools. This blog post is all about diving into profiling a Triton GPU kernel, with a specific focus on compute performance, using the powerful...

Intelligent inference request routing for large language models

Today's AI environment is experiencing a surge in specialized Large Language Models (LLMs), each possessing unique abilities and strengths123: Some are strong in reasoning and mathematics, while others may excel in creative writing. Yet most applications resort to a...

Enhancing AI inference security with confidential computing: A path to private data inference with proprietary LLMs

Red Hat's Office of the CTO is collaborating in the upstream project Tinfoil community to explore pioneering a complete, cloud-native solution for Confidential AI. The community is focused on solving one of the toughest AI security challenges facing the enterprise:...

Welcome to Red Hat Emerging Technologies

Here you’ll find information about the emerging technology projects the Red Hat Office of the CTO is working on. For us, “emerging technologies” refers to those technologies that are still taking shape in the enterprise or even in research communities. Emerging technologies engineering is pre-product, purely upstream work. Not everything you see here will become part of the Red Hat portfolio roadmap, but we want to share this information with our customers and partners and encourage your participation. At Red Hat, we love to co-create with open source communities, partners and customers. We look forward to you engaging with us and providing your ideas and feedback on these projects.