From zero to benchmark: Deploying LLM inference on CPUs with Red Hat AI Inference

What if your standard-issue CPU infrastructure could serve as a hub for enterprise AI? With the right tools, it can. You can deploy a large language model on CPU inside a container on Red Hat Enterprise Linux (RHEL), query it via an OpenAI compatible REST API, and...

Benchmarking enterprise LLMs on AMD GPUs with Red Hat AI Inference and Podman

Deploying large language models (LLMs) on enterprise hardware requires a balance of reproducibility, security, and performance. Accurately benchmarking model performance under production-like workloads is both challenging and crucial for optimizing infrastructure...

Beyond guardrails: mitigating prompt injection attacks

A primary reason many AI pilot projects fail to reach production is lack of confidence in the security of the systems. One of the most common and pernicious security issues affecting generative AI (GenAI) systems is the prompt injection attack—now listed by OWSAP as...

Welcome to Red Hat Emerging Technologies

Here you’ll find information about the emerging technology projects the Red Hat Office of the CTO is working on. For us, “emerging technologies” refers to those technologies that are still taking shape in the enterprise or even in research communities. Emerging technologies engineering is pre-product, purely upstream work. Not everything you see here will become part of the Red Hat portfolio roadmap, but we want to share this information with our customers and partners and encourage your participation. At Red Hat, we love to co-create with open source communities, partners and customers. We look forward to you engaging with us and providing your ideas and feedback on these projects.