Additional Services of Red Hat OpenStack
Estimated reading time: 12 minutes.
- Objective
-
Describe additional OpenStack services useful to applications and their virtual machines.
OpenStack Support for Cloud-Native and Other Kinds of Applications
It is not uncommon to use OpenStack (and public cloud platforms, for that matter) to just replicate, in a virtual infrastructure, the same servers, networks, etc you would manage in a data center with physical servers and no virtualization. For some organizations, just switching from days or weeks of provisioning physical machines to minutes of provisioning virtual machines (as long as your cluster has the capacity to spare) is already enough benefit. These organizations might be happy with just the fundamental OpenStack services.
But many benefits from using private clouds come from designing applications to take advantage of the dynamic, on-demand infrastructure that cloud platforms can provide, and from automating provisioning and management of such applications.
Even if an organization is not yet designing applications and their operational processes to be cloud-native, the fact that provisioning basic server, networking, and storage infrastructure became faster enables IT teams to focus on other non-functional requirements, such as security and horizontal scalability. OpenStack helps address those requirements.
Additional OpenStack Services for Storage
Besides Cinder, Red Hat OpenStack Services on OpenShift offers two other services to store data:
-
Manila: Shared Filesystems service
-
Swift: Object Store service
Almost all workloads use Block storage from Cinder because server instances need virtual disks, but those virtual disks are not sufficient for all kinds of workloads. Some workloads require shared file systems, and the use of networked file-sharing services, such as NFS, is very common for scaling applications to multiple servers or integrating different applications by sharing data files.
Public cloud started without file-sharing services, but offered object storage as an alternative, which made it very popular among cloud-native applications and on the business backends of mobile applications. Object storage also enables scaling applications to multiple servers and integrating different applications by sharing object files.
For end users, object storage may look just like file storage: buckets or containers are folders or directories, and objects are files. But, from a developer’s perspective, they are very different. For example, objects support write-once semantics and versioning, but files support append and random access modes.
In the end, applications are typically designed to use either of them (file or object storage) and cannot be easily configured to use the other. This course is not intended to provide a comparison of the benefits and capabilities of object versus file storage: from the perspective of OpenStack Operators, the decision was already made, and their job is to provide whatever of these capabilities applications require.
OpenStack File Shares: Manila
Manila offers access to remote file shares using a variety of network protocols, according to the capabilities of the storage backends configured on your OpenStack cluster. Typical business requirements, such as high availability of data by replication, depend on the storage backend rather than on OpenStack.
The role of Manila is to provide a uniform interface for making file shares available to application VMs, so an OpenStack Operator does not need direct access to the storage backends themselves and their proprietary administration interfaces. Depending on the backend, Manila may also abstract networking and authentication details, making it look like the file share exists inside the virtual tenant network.
Manila does not abstract the file-sharing protocol itself: if an application VM expects a remote NFS share, Manila must be configured to provide NFS services. Besides, it is outside the scope of Manila to abstract file system security, such as uid, gid, and permission masks of POSIX files.
OpenStack Objects: Swift
Swift offers object storage capabilities similar to Amazon S3. S3 is a de-facto standard, supported by a large number of storage vendors and applications, but there is no formal specification nor interoperability test suite to show objectively if two S3 products from different vendors are similar or different in capabilities. Unfortunately, there is no way any vendor can tell, except by extensive testing, if one application works fine with the S3 API implementation of a specific storage vendor.
Swift has its own application API, but it never got close to the popularity of Amazon S3. Most storage backends certified for Red Hat OpenStack offer both the Swift and S3 APIs, and an OpenStack Operator can manage object storage using Swift while applications access that storage using S3. Which is already good because most storage vendors do a good job of emulating the S3 API from the perspective of applications but require their own proprietary administration interfaces, so Swift can abstract these interfaces for an OpenStack Operator.
Swift is one of the few services in an OpenStack cluster that is accessed directly by end-user applications running outside its cluster. Unlike traditional networked file systems, which need special configurations at the end-user operating system and require application protocols not designed for remote networks and the internet, object storage uses the HTTP protocol which mobile and web applications commonly use to interact with other cloud services.
Additional OpenStack Services for Networking
Besides Nova, two services of Red Hat OpenStack Services on OpenShift offer networking capabilities:
-
Octavia: Load Balancer service
-
Designate: DNS service
These services enable dynamic and scalable applications, not tied to manually configured IP addresses and DNS names: New applications can be immediately accessible using a human-readable name, and multiple servers accessed using a single IP address. If used together, the human-readable name, or vanity name, can point to a single IP address.
OpenStack Network Load Balancing: Octavia
Octavia enables fronting a server group or server pool with a single IP address, and forwarding network requests to any server of the group. This enables scalable and high-available applications, because more servers can handle more load, and servers can run in different compute nodes, insulating application users from failures of those compute nodes and of individual server instances.
Octavia also works with Nova to enable the autoscaling of server groups, so the number of servers grows and shrinks according to demand. Octavia can work on both Layer 3, as a pure IP load balancer, or at Layer 7, as an HTTP and TLS load balancer. In either case, Octavia can detect failures internal to server instances, such as when an application becomes unresponsive, and stops sending sending traffic to those instances.
OpenStack Dynamic DNS: Designate
The DNS protocol is one of the oldest standards of the Internet, but the management of DNS servers is totally proprietary. Octavia offers OpenStack Operators with a standard interface for managing DNS zones and records, and also integration with Nova, Neutron, and Octavia, to automatically assign DNS names to resources such as server instances, floating IP addresses, and load balancers.
Additional OpenStack Services for Automation
In the traditional data center world, each server is unique. Once you install all software and perform all configuration changes on that server, it’s time-consuming and error-prone to replicate the same state in another server. In the cloud-native world, IT personnel strives to automate those configurations so it’s easy and fast to reproduce the state of a server to the point that it is ready for its end-users.
To support the automation of workloads, Red Hat OpenStack Services on OpenShift adds to Nova and Glance the following services:
-
Heat: Orchestration service
-
Placement: Placement service
The first service (Heat) automates the creation and management of many OpenStack API resources, such as servers, networks, volumes, and load balancers, required to deploy complex application workloads. The second (Placement) addresses the distribution of those resources, especially servers, over different compute nodes to address performance and reliability requirements.
OpenStack Workload Orchestration: Heat
OpenStack Glance provides a first step towards reproducibility, by enabling storage and reuse of server images to create multiple clones of the same server. Nova’s support for cloud-init provides another piece, by enabling each of those servers to get unique configurations, such as different static IP addresses and SSH keys.
However application workloads are rarely contained in a single server instance. Most real-world applications are composed of multiple VMs, for example, the traditional three-tier architecture of presentation, business, and persistence, which could be realized as:
-
Presentation: Web server with static HTML pages and JavaScript code.
-
Business: Java application server, or web server running REST API endpoints using Java, Node.js, Ruby, or whatever is your programming language runtime of choice.
-
Persistence: A relation or NoSQL database server.
It is also very common that each of those tiers consists of multiple servers, for example, a web server farm or replicated primary (read-write) and secondary (read-only) database instances.
Those server instances need network connectivity, firewall rules, load balancers, and human-readable names for their end-user web pages and REST API entry points. Manually creating and configuring all OpenStack API resources required by such applications can be a daunting task!
OpenStack Heat enables defining all those resources using a Heat template with YAML syntax. Heat also maintains the relationship between templates and the resources created from them, so these resources can be changed and deleted from the templates instead of manually using the OpenStack APIs for each resource type.
Heat templates can be parametrized, enabling reuse of the same template in different scenarios, such as different user-facing application URLs, different IP addresses for external resources, and different sizes of server instances and storage volumes.
OpenStack Server Scheduling: Placement
The OpenStack Placement service is different than other services introduced so far because OpenStack Operator users and external applications do not interact directly with it, but indirectly through resources of other services, especially Nova.
The Placement service tracks compute resource availability and consumption throughout an OpenStack cluster, and enables other OpenStack services to define and track different classes of resources. For example, Nova defines a number of CPU cores and memory of compute nodes, and server instances consume those resources. Neutron defines a number of IP addresses of internal virtual networks and of floating IP addresses available for connectivity with external networks. Cinder defines the total disk space and number of logical units (LUNs) of each storage backend.
OpenStack API resources can consume compute resources from different OpenStack services. For example, server instances consume CPU and memory from compute nodes, IP addresses from virtual networks, and disk space from a storage backend. If there are no sufficient compute resources available in the cluster, you cannot create the API resource.
Thanks to Placement, Nova can schedule each server instance to a compute node with sufficient available compute resources for that instance.
Placement also enables the definition of new classes of compute resources, for example: the availability of GPUs in compute nodes. The server instances can request a number of GPUs, and an OpenStack Operator can be sure that instance either gets the GPUs it needs, or fails to start.
Defining new classes of compute resources is usually an OpenStack Administrator task, while configuring workloads to consume compute resources from those classes is an OpenStack Operator task.
Additional OpenStack Services for Security
We already know that security in OpenStack starts with Keystone, as the authentication entry point for all OpenStack services. While most end-user applications do not need to invoke OpenStack APIs, and thus would have no need for direct interactions with Keystone, most applications do interact with a wide range of services. These services range from external APIs of a software vendor to middleware running on other servers inside and outside of their OpenStack cluster, such as database servers and messaging middleware.
Management of credentials to access those services and APIs is a potential source of security issues. They should not be hardcoded in either the application source code or on its configuration files. Ideally, they would live outside of application VMs. Security-conscious organizations require that applications use Key Management Services (KMS) to store such credentials, and OpenStack offers Barbican: the Key Management service.
OpenStack Secrets Management: Barbican
OpenStack Barbican enables secure storage of many types of keys, including symmetric and asymmetric keys, and including both passwords and certificate files. Applications authenticate to Barbican using Keystone to retrieve keys, and OpenStack Operators manage those keys independently of applications.
Multiple OpenStack services offer integration with Barbican, for example, to encrypt data stored in Cinder volumes or Swift objects.
Operation and Administration of Additional Services
It is expected and recommended that most Red Hat OpenStack Services on OpenShift clusters do enable all of the additional services introduced here, and that OpenStack Operators encourage application developers and application administrators to take advantage of them.