How Open Data Hub learners become the teachers

The prospect of true machine learning is a tangible goal for data scientists and researchers. It has been long known that the platform on which such ML apps can run have to be fast and hyper efficient so that learning can be that much faster. This is the motivation for Red Hat engineers in the Office of the CTO who are working to optimize such an open source platform: Open Data Hub.

Open Data Hub is built on Red Hat OpenShift Container Platform, Ceph Object Storage, and Apache Kafka/Strimzi integrated into a collection of open source projects to enable a machine-learning-as-a-service platform. That’s a lot of components to be integrated, and to ensure that their contributions to Open Data Hub perform well, Red Hat engineers have taken the step of creating an Internal Data Hub within Red Hat as a proving ground and learning environment.

Continue reading “How Open Data Hub learners become the teachers”

Exploring Unsupervised Deep Learning

The concept of artificial intelligence, which seemed so much like science fiction a few decades ago, has made real, practical inroads in producing results that organizations can find useful. What’s making those results happen, though, isn’t esoteric pie-in-the-sky theory: it’s creating statistical models that have been trained to make decisions. And trained a lot.

Artificial intelligence itself is a term that, for now, has had less of a focus than the more results-oriented machine learning, where a computer system is given input and output data and then is directed to infer the mathematical rules that govern the transformation of that data.

“It’s like pointing a program to look at the solar system and then have it figure out the laws of motion that govern a planetary system,” explained Sanjay Arora.

Continue reading “Exploring Unsupervised Deep Learning”

Merging Research and Software with Open Source

(There’s a great  new conference in the U.S., DevConf.US, returning in 2019 to Boston University (15 to 17 Aug). This highly-technical conference is interested in drawing a diverse group of speakers and attendees, with a specific emphasis on people who are new to speaking and tech conferences in general. Only in its second year, DevConf.US builds on the successful decade-spanning run of DevConf.CZ in Brno, CZ.

This is a session from DevConf.US 2018. The call for proposals to present at DevConf.US 2019 is now open.)

Software development has found a niche in almost every aspect of our transactional lives, be it retail, finance, and even academia. This last sector is a particularly strong growth area in the past few years, as more and more coders are looking at universities and colleges as a direct career path.

This isn’t just software for supporting faculty, staff, and student operations (though that’s important too). According to Dr. Andrei Laptets, Associate Professor at Boston University, it also includes software for any scientist and researcher who needs to manage and analyze a wide variety of data-driven projects.

Continue reading “Merging Research and Software with Open Source”

Hardware Isolation in the Cloud – HIL on the Mass Open Cloud

The Mass Open Cloud (MOC) is an open cloud exchange that provides compute resources to university researchers. The virtualization infrastructure is built on Red Hat OpenStack Platform, using Foreman for provisioning and Ceph for distributed storage. But the MOC has also developed its own tools to make bare metal computing available. We talked to Naved Ansari, one of the MOC developers, about some of these developments.

In a typical cloud computing environment, users are provided with a virtual machine running on the same physical machine as other virtual machines. This is a way to maximize compute resources, by not leaving too many machines unused. Virtual machines work well for a lot of workloads, but occasionally people need access to bare metal without a virtualization layer.

Continue reading “Hardware Isolation in the Cloud – HIL on the Mass Open Cloud”

Machine Learning as a Service

Experimenting with machine learning algorithms or integrating such techniques into an existing environment often presents challenges, like selecting and deploying the right infrastructure, in addition to having the necessary data science background and skills, etc. In this post, we present a service that allows users to train machine learning models, run analyses using trained models, as well as manage data required for such models or analyses. Now machine learning models or the prediction results can be easily integrated in to existing continuous integration (CI) or IT infrastructure using REST API.

Overview

The main components of such a service are Apache OpenWhisk, Red Hat OpenShift, and Ceph Storage. These components are available under AI Library at https://gitlab.com/opendatahub/ai-library. OpenWhisk is a serverless computing platform that provides the interface through which users can submit HTTP requests to train or execute machine learning models. HTTP requests submitted to OpenWhisk are actually targeted to stateless functions, called actions, that run on the platform. Ceph Storage is used for storage of training and prediction data, models and results. Users can submit data in to Ceph backend through OpenWhisk actions provided in our implementation (s3.py) or any custom tools such as RADOS object storage utility that can interact with Ceph storage. The action ‘s3.py’ not only supports Ceph, but also any S3-compatible storage backend.

Continue reading “Machine Learning as a Service”

UKL: A Unikernel Based on Linux

Unikernels are customized, single address space bootable images composed of an application and the required bare-minimum kernel functionality. Today’s unikernels have demonstrated substantial performance and security advantages over monolithic and microkernels, but none have yet achieved widespread adoption.
The fundamental problem is that today’s unikernels, which have been developed by forking existing operating systems or as clean-slate designs, have abandoned the evolutionary community process that has made Linux such a success. In this post we describe an alternative approach we are pursuing with the goal of making unikernels a community supported, evolving capability of Linux and and the GNU C LIbrary (glibc).

Continue reading “UKL: A Unikernel Based on Linux”

A Hub for Open Data at Mass Open Cloud

Open source software is good. Open source plus open data is even better. That makes initiatives such as the Open Data Hub both useful in and of themselves and as a template for maintaining control over your data.

Access to, and the ability to collaboratively build upon, open source code is genuinely useful. If it weren’t, open source software wouldn’t have become such an important part of how technology has developed over the past couple of decades. There are ideological reasons to prefer open source as well, but its effectiveness as a development model has won over the pragmatists.

Continue reading “A Hub for Open Data at Mass Open Cloud”

Malleable Metal – Integrating SAN-booting with Foreman

The world of multi-tenant bare metal cloud computing in the datacenter is increasingly important.  With tenants being offered their own servers rather than locked-down VMs or compute services, the potential for innovation is much higher.  Mass Open Cloud aims to offer a multi-tenant cloud where hardware would be shared between organizations, such as universities, with tenants able to access bare metal instances directly. Here’s how we propose to create a standardized architecture to provide a seamless elastic bare-metal experience for Mass Open Cloud and similar environments.

Our solution to the bare-metal-as-a-service problem combines two projects: Mass Open Cloud’s Malleable Metal as a Service (M2) and the Red Hat stewarded Foreman Project.  Where M2 provides the means for provisioning servers, Foreman provides the orchestration and user interface.

Continue reading “Malleable Metal – Integrating SAN-booting with Foreman”

Introducing now + Next

As an industry we look to open source communities as our core innovation engine.  At Red Hat we’re always monitoring, participating in, and even creating these open source communities. Here’s how you can garner some insight into where the industry, and Red Hat, might be going next.

Continue reading “Introducing now + Next”

  • Page 1 of 2
  • 1
  • 2
  • >