Building a Scalable TensorFlow Twitter Bot for Red Hat Summit

Red Hat’s AI Center of Excellence and PerceptiLabs wanted a way to demonstrate a TensorFlow model to the public during the 2019 Red Hat Summit. The plan was for this model to take images as input, and then respond with the likelihood of a Red Hat fedora being in that image. Here’s what we learned during Red Hat Summit.

This application, which we called Fedora Finder Bot, would be featured during Red Hat CTO Chris Wright’s keynote, where PerceptiLabs demoed their AI platform.

Our initial solution for this objective would be a Twitter bot that receives tweets or direct messages and replies with the output from the TensorFlow model. Twitter being a public service, we felt it could make the model available to a large number of users, so that any user could just tweet to the bot with a picture and the bot would respond with the model’s output.

Continue reading “Building a Scalable TensorFlow Twitter Bot for Red Hat Summit”

Rook Changes the Kubernetes Storage Landscape

It’s no secret that if you want to run containerized applications in a distributed way, then Kubernetes is the platform for you. Kubernetes’ role as an orchestration platform for containers has taken center stage to become a main player for automating deployment, scaling, and management of applications within containers. Red Hat’s own OpenShift Container Platform is a Kubernetes distribution that uses Kubernetes optimized for enterprises.

Storage has been one of the areas of potential optimization. Many containers, by their very nature, are usually small enough to be easily distributed and managed. Containers hold applications, but the data those applications use needs to be held somewhere else, for a number of reasons. Of particular interest in this post, we want to avoid the containers themselves becoming too large and unwieldy to be effectively managed.

Continue reading “Rook Changes the Kubernetes Storage Landscape”

Consumption is Fractal: Open Source Sustainability

One of the more obscure terms one might hear bandied about in the free and open source software ecosystem is the so-called “bus factor.” The somewhat-informal term refers to the state of a given project based on its sustainability.

Specifically, bus factor is shorthand for the question: what would happen to your open source project if one of your community members were hit by a bus? Would the project survive? Or is so much workflow and institutional knowledge wrapped up in that one person that your project would be damaged, possibly to the point of no recovery?

Continue reading “Consumption is Fractal: Open Source Sustainability”

Anomaly Detection on OpenStack Logs Using Machine Learning

(There’s a great  new conference in the U.S., DevConf.US, returning in 2019 to Boston University (15 to 17 Aug). This highly-technical conference is interested in drawing a diverse group of speakers and attendees, with a specific emphasis on people who are new to speaking and tech conferences in general. Only in its second year, DevConf.US builds on the successful decade-spanning run of DevConf.CZ in Brno, CZ.

This is a session from DevConf.US 2018. The call for proposals to present at DevConf.US 2019 is now open.)

In this session from the CentOS Dojo held as part of DevConf.US, OpenStack technical support engineers Madhur Gupta and Shatadru Bandyopadhyay talk about how to use machine learning for anomaly detection on OpenStack logs. Once an anomaly is detected in the logs, it can be used to automate further action, while helping in root cause analysis.

The challenge with anomaly detection in OpenStack in the first place is that it generates a significant quantity of logs, even in relatively simple production setups. How do you ingest and detect anomalies in all that data?

Continue reading “Anomaly Detection on OpenStack Logs Using Machine Learning”

Merging Research and Software with Open Source

(There’s a great  new conference in the U.S., DevConf.US, returning in 2019 to Boston University (15 to 17 Aug). This highly-technical conference is interested in drawing a diverse group of speakers and attendees, with a specific emphasis on people who are new to speaking and tech conferences in general. Only in its second year, DevConf.US builds on the successful decade-spanning run of DevConf.CZ in Brno, CZ.

This is a session from DevConf.US 2018. The call for proposals to present at DevConf.US 2019 is now open.)

Software development has found a niche in almost every aspect of our transactional lives, be it retail, finance, and even academia. This last sector is a particularly strong growth area in the past few years, as more and more coders are looking at universities and colleges as a direct career path.

This isn’t just software for supporting faculty, staff, and student operations (though that’s important too). According to Dr. Andrei Laptets, Associate Professor at Boston University, it also includes software for any scientist and researcher who needs to manage and analyze a wide variety of data-driven projects.

Continue reading “Merging Research and Software with Open Source”

Hardware Isolation in the Cloud – HIL on the Mass Open Cloud

The Mass Open Cloud (MOC) is an open cloud exchange that provides compute resources to university researchers. The virtualization infrastructure is built on Red Hat OpenStack Platform, using Foreman for provisioning and Ceph for distributed storage. But the MOC has also developed its own tools to make bare metal computing available. We talked to Naved Ansari, one of the MOC developers, about some of these developments.

In a typical cloud computing environment, users are provided with a virtual machine running on the same physical machine as other virtual machines. This is a way to maximize compute resources, by not leaving too many machines unused. Virtual machines work well for a lot of workloads, but occasionally people need access to bare metal without a virtualization layer.

Continue reading “Hardware Isolation in the Cloud – HIL on the Mass Open Cloud”

Red Hat’s Open Source AI Vision

Analytics, Machine Learning, and AI represent a fundamental transformation that over the coming decade will affect every aspect of society, business, and industry. It will fundamentally change, how we interact with computers – and how we develop, maintain, and operate systems. It’s impact will be visible in our part of the universe much sooner than for the analog world. This deeply affects both open source in general, as well as Red Hat, its ecosystem, and customer base.

In this video from the inaugural DevConf.US 2018, Daniel Riek who leads the AI Center of Excellence in Red Hat Office of the CTO, talks about this coming change.

Continue reading “Red Hat’s Open Source AI Vision”

Transforming IT Operations: A Roadmap

Digital transformation is more than just a fancy buzzword. With 85 percent of Global 2000 CEOs believing in digital innovation as a driver of business success, it is estimated that nearly $2.1 trillion will be invested in digital transformation technologies in 2019.

According to Mary Johnston Turner, Director, Management Software BU Evangelism,  the drivers to digital transformation are going to play a significant role in driving IT decision-making for the near-term future. Turner outlined the significant driving factors in her 2018 Summit breakout session “Transforming IT Ops: The future of IT automation & management.”

Continue reading “Transforming IT Operations: A Roadmap”

Machine Learning as a Service

Experimenting with machine learning algorithms or integrating such techniques into an existing environment often presents challenges, like selecting and deploying the right infrastructure, in addition to having the necessary data science background and skills, etc. In this post, we present a service that allows users to train machine learning models, run analyses using trained models, as well as manage data required for such models or analyses. Now machine learning models or the prediction results can be easily integrated in to existing continuous integration (CI) or IT infrastructure using REST API.

Overview

The main components of such a service are Apache OpenWhisk, Red Hat OpenShift, and Ceph Storage. These components are available under AI Library at https://gitlab.com/opendatahub/ai-library. OpenWhisk is a serverless computing platform that provides the interface through which users can submit HTTP requests to train or execute machine learning models. HTTP requests submitted to OpenWhisk are actually targeted to stateless functions, called actions, that run on the platform. Ceph Storage is used for storage of training and prediction data, models and results. Users can submit data in to Ceph backend through OpenWhisk actions provided in our implementation (s3.py) or any custom tools such as RADOS object storage utility that can interact with Ceph storage. The action ‘s3.py’ not only supports Ceph, but also any S3-compatible storage backend.

Continue reading “Machine Learning as a Service”

Blockchain: A Primer on How to Identify Good Use Cases

Everyone has an opinion on how Blockchain will change business and society. Quite a few startups are working on their Blockchain-based products or services, and some of them are even using initial coin offerings (ICOs) as a funding vehicle. However, it’s hard to find good use cases that haven’t been solved already with more traditional technologies and business models.

To overcome this hurdle  I have created a simple framework that might help people to evaluate use cases and identify the most promising ones. There are many things to take into account when evaluating a Blockchain use case but there are few that are crucial, the others can be considered implementation details. We need to begin with the most important one: what class of problems Blockchain is designed to address.

Continue reading “Blockchain: A Primer on How to Identify Good Use Cases”