Cloud-native software development with Virtual Application Networks

Communication between distributed software components in a cloud-native application is an important and challenging aspect of cloud-native development. This post introduces a solution to that problem using Virtual Application Networks (VANs).  A VAN can be set up by a developer and used to connect the components of an application that are deployed in different public, private, and edge cloud environments.

Cloud-native development is about writing software in such a way that it can be deployed easily, flexibly, and automatically into the hybrid-cloud ecosystem to take advantage of the scale of the cloud.  A big part of taking advantage of cloud scale is the ability to deploy components of a distributed system in different locations.

Continue reading “Cloud-native software development with Virtual Application Networks”

Data integration in the hybrid cloud with Apache Spark and Open Data Hub

In this post we introduce the basics of reading and writing Apache Spark DataFrames to an SQL database, using Apache Spark’s JDBC API.

Apache Spark’s Structured Streaming data model is a framework for federating data from heterogeneous sources. Structured Streaming unifies columnar data from differing underlying formats and even completely different modalities – for example streaming data and data at rest – under Spark’s DataFrame API.

Continue reading “Data integration in the hybrid cloud with Apache Spark and Open Data Hub”