Red Hat OpenShift to Develop and Deploy Enterprise ML/DL Applications
Investment is significantly increasing in machine learning and deep learning (ML/DL) to create value with different objectives such as masking complexity, automatization, reduce cost, grow businesses, better serve customers, discoveries, research and innovation, etc. Strong open-source communities for ML/DL on Kubernetes and Openshift were created and evolving. These communities are working to allow data scientists and developers access and consume ML/DL technologies. Working on our local computer and putting ML/DL models into production requires navigating across a vast and complex space. Where do we deploy our code for training, where do we deploy our code for batch or online inferences. There are situations where we will need to deploy our machine learning workflows on a multi-architecture environment, our on a hybrid cloud environment. Today’s data centers are made up of heterogeneous systems (x86, IBM Power Systems, IBM Z, High Performance Computing, accelerators such as GPUs or FPGAs …), running heterogeneous workloads, with specialized ML/DL frameworks, and each with its strength. In addition to that, we can see the cloud in all its dimensions (Public and Private Cloud, Hybrid Cloud, Multi-Cloud, Distributed Cloud). We can for instance have a database with critical data running on an IBM Power Systems that we want leverage for our models, run our training code using GPUs, deploy batch or online inference on IBM Z / LinuxONE where critical transactional applications are and avoid latency, another inference on a cloud or at the edge. There is an important number of options to consider depending on your business. A typical ML/DL workflow starts with a business objective and involves a design thinking to understand users, challenge assumptions, redefine problems and co-create (for instance putting IT and Data Scientist teams in the same room) a solution to prototype and test by iteration. We then collect private and public data, refine, and store the data, create, and validate models until we put everything into production for the real world. We need to think about the scalability of the application, the resilience, versioning, security, availability, etc. This requires additional expertise, often specialized hardware resources increasing the need for resource management and utilization. The data scientists cannot manage the entire process that can be complex and for those I personally know, they want to have access to performant hardware, be focus on the data and the creation of models. This is also why we can see deep work not being completely exploited because it is not prepared for production. This is where containers and Kubernetes can avoid this kind of situation by accelerating ML/DL adoption and break all these barriers. There is a clear movement embracing Linux containers and Kubernetes to develop ML/DL applications but also to deploy them. Containers and Kubernetes are the way to simplify the access to underlying infrastructure by masking the complexity, allow to manage the different workflows such as development or application lifecycle. Red Hat OpenShift will provide additional capabilities which are well suited for enterprise environments
What is OpenShift ?
Kuberntes is an open source project and Red Hat OpenShift is a certified Kubernetes platform and distribution. It’s a container application platform based on Kubernetes for enterprise application development and deployment. Red Hat is one of the top contributors to the Kubernetes community. OpenShift is a family of containerization software such as OpenShift Online, OpenShift Dedicated, or OpenShift Container Platform which is an on-premise platform as a service with Docker containers orchestrated and managed by Kubernetes on Red Hat Enterprise Linux. OKD is the open source version of OpenShift, known until August 2018 as OpenShift Origin (Origin Community Distribution). It is the upstream community project. The idea behind OpenShift is really about enhancing the management and developer experience for deploying enterprise (at-scale) applications on Kubernetes.
What is the difference between OpenShift and Kubernetes ?
Kubernetes has many distributions relies mostly on communities or external expertise’s if we need some help. Companies often ask for official support, especially if you run critical business, but not only. In order to fully operationalize a Kubernetes environment and run containerized applications across a distributed system environment, we need more than just Kubernetes expertise when your objective is to deploy enterprise applications on Kubernetes. We need to take into account a number of things such as a robust security, a developer-friendly environment, cluster management, integrated builds and CI/CD services, multi-architecture and multi-platform deployment wherever it is in your data center, public cloud, multi-cloud, the edge, virtual machines, bare-metal, x86, IBM Power Systems, IBM Z, registries to deploy images, automatization of operations, secure container images, management and automatic container updates, manage hybrid storage, multitenancy and multiple clusters management, etc. The topic of security is really a key element, and if we can avoid a headache by having a more secure foundation by default it will certainly help. The default policies on Red Hat OpenShift are more strict than on Kubernetes. For instance, Role-Based Access Control (RBAC) is an integral part of OpenShift. It’s maybe okay to use Kubernetes without RBAC security for a small dev/test setup but when real life and production come it is necessary to have some level of permissions. Kubernetes is like a Linux kernel. We need more than just a Linux kernel, we need a Linux platform distribution to run Linux applications. In practice, for those that already installed and played with Kubernetes, Red Hat OpenShift supports the use of kubectl and users can use the native Kubernetes command-line interface. We can also use the command line for developers and take advantage of additional capabilities with other command line tools such as oc which is an equivalent if Kuberntes’s kubectl but with some differences such as the possibility to build a container image from a source and deploy it into environments with a single command or odo to write, build and debug applications on a cluster without the need to administer the cluster itself. Red Hat OpenShift supports Kubernetes Operators and Deployments, third-party tools such as Helm Charts (application deployment), Prometheus (Monitoring and Alerts management), Istio (manage a distributed microservice architecture), Knative (serverless), Internal Container Registry, Logging stack based on EFK (ElasticSearch, Fluentd, Kibana) or Jenkins that makes it easy to deploy our applications with CI/CD pipelines. We can use a single account to authenticate making permissions management easier. One great thing of Red Hat OpenShift is the management of container images with Image Stream allowing for example to change a tag for an image in a container registry without downloading the image, tag it locally and push it back. With OpenShift, once you uploaded the image, you can manage it within OpenShift it’s virtual tag. It’s also possible to define triggers which starts deployment for example when tag changes its reference (e.g. from devel to stable or prod tag) or if a new image appears. Another difference between Kubernetes and OpenShift is the web-based user interface. The Kubernetes dashboard is needed to be installed separately and we can access it via Kube proxy. Red Hat OpenShift’s web console has a login page, easy to access and much helpful for daily administrative work as you can create and change of resources via a form. We can install a Red Hat OpenShift clusters in the cloud using managed services (Red Hat OpenShift on IBM Cloud, Red Hat OpenShift Service on AWS, Azure Red Hat OpenShift) or we can run them on our own by installing from another cloud provider (AWS, Azure, Google Cloud, Platform agnostic). We also have the possibility to create clusters on supported infrastructure (Bare Metal, IBM Z, Power, Red Hat OpenStack, Red Hat Virtualization, vSphere, Platform agonistic) or a minimal cluster on our laptop which is useful for local development and testing (MacOS, Linux, Windows).
Why Red Hat OpenShift for ML/DL ? To build a production-ready ML/DL environment
I think everybody would agree that creating performant ML/DL models and deploying ML/DL in production requires different set of skills. To allow a deployment of a ML/DL application in production, we need to put in place an iterative process starting from setting the business goals, gathering and preparing the data, developing models, deploying models, inferencing, monitoring to managing accuracy over time. In order to execute this process, we need to implement a ML/DL architecture with ML/DL tools, DevOps tools, data pipelines, access to resources (compute, storage, network) whether on a private/public/hybrid/multi-cloud environments. Red Hat OpenShift is making a difference because it allows data scientist / developers to focus on their model / code and deploy it on Kubernetes without a need to learn Kubernetes in deep. We automate once, and then we just develop whether the environment. In other words, it’s possible to manage the complexity of ML/DL model deployments and democratize access to the techniques allowing the deployment of any containerized ML/DL stack at scale on any environment. Easy to access specialized hardware resources such as GPUs, FPGAs, x86, IBM Z or IBM Power Systems and easy to manage lifecycle of hardware drivers and libraries.
Conclusion
Red Hat OpenShift has many features and benefits that can help data scientists and developers to really focus on their business and use the tools and language they are most comfortable with. OpenShift brings additional security controls in place and the tools to manage multiple applications (multitenancy environment). It makes all environments much easier to manage.
