The National Institute of Standards and Technology (NIST) defined the most important terms regarding the cloud. Whenever one of these terms is used throughout the article, it follows the definition provided by the NIST.

Motivation

The cloud is omnipresent. On every conference, in every journal and even while reading the newspaper one will stumble upon this buzzword. Whether as cause or as effect is not provable anymore, but all major IT companies offer products or services for the Cloud [1, 2, 3] (or even a Cloud itself [4, 5, 6]). However, customers are still very cautious in implementing their own private cloud or migrating their application into the public cloud. One of the biggest fears besides properties as privacy and security is the implicit vendor lock-in.

Vendor Lock-In

Currently cloud providers focus on adding new services, adding unique features and integrating/coupling their offerings. Since there is no universal agreement on what a cloud provider needs to offer each one of them creates its own profile visible to the market. Most offerings target a certain group of customers. Once a customer is won, it is hard for him to change to another provider. All specific features the customer is already using prevent him from easily migrating his application to a new cloud provider. Amazon for example published 160 new features or services in 2012 underpinning its position as one of the market leaders.

Hybrid Cloud

Another drawback of diverging cloud technologies is the growing effort needed to combine cloud offerings. This hybrid cloud approach enables a customer to keep his most sensitive data in his private cloud and only leverages the resources of a public cloud for tasks that do not operate on said data. Not only has the application to work with two different APIs to manage the private and public cloud but probably has to incorporate two completely different concepts.

What the Customer Wants

From the customer’s point of view it is only consequent to try to integrate as many different services from different providers as they all vary in cost, functionality and quality. To get the best, fastest and cheapest solution possible, it is unacceptable to be limited to only the offerings of a single provider. However, this is where the road leads today. The additional complexity of finding and integrating the right services weighs a lot heavier on the development budget when compared to established ready-to-use cloud environments.

This can only change with interoperability and portability of cloud services backed by an industry standard. The following paragraphs will try to provide an overview over tools that try to ease the pain of working with different clouds, APIs that hide differences between products and standards trying to solve the problem once and for all (really?).

Tools

This section is not intended to be an in depth usage analysis of the mentioned tools nor does it compare competitors. Each product shall merely be an example for a class of tools solving a specific problem.

OpenStack

OpenStack is  a software to build your own (private) cloud. The project is backed by many big companies and institutions like IBM, HP, Dell, SUSE and CERN to name only a few of them.  Once deployed it offers the capabilities to manage virtualized compute, storage and network resources. It has a dashboard so no administrator or user needs to understand the details of a hypervisor or a network management system. All functionality is also accessible through an API. The community behind OpenStack is very strong and growing, maybe one of the reasons why IBM, CALXEDA and SUSE are already offering products/services powered by OpenStack.

Source www.openstack.org

Launching an instance on OpenStack (Source www.openstack.org)

(Yes, there are Eucalyptus, OpenNebula, CloudStack and others that do pretty much the same but don’t seem to be as popular right now.)

Apache Libcloud

Apache Libcloud is a Python module that gives administrators and programmers a uniform interface to access different IaaS providers. There is a long list of supported providers and features. See following code snippet for an example of listing all nodes of three different providers (Amazon EC2, Slicehost and Rackspace).

 (Source http://libcloud.apache.org/getting-started.html)

The big advantage is that one has not to worry about the API differences between the providers. This makes switching the provider a lot easier and eliminates the need of becoming an API expert before being able to switch to a new provider. This naturally comes with a downside: not all specialized features are available through the uniform interface and it takes some time until Libcloud implements and releases API changes.

Deltacloud

Deltacloud is similar to Libcloud, but is implemented in Ruby and not in Python. There is also a vast amount of supported ‘Clouds’. The major difference is that Deltacloud also supports a CIMI mode (see Cloud Infrastructure Management Interface in the next chapter Standards).

Standards

The respective paragraphs outline the scope of the standards but, by far, don’t resemble a full analysis.

Open Virtualization Format (OVF)

The specification of the Open Virtualization Format (OVF) is maintained by the Distribute Management Task Force (DMTF). According to the spec it

“[…] describes an open, secure, portable, efficient and extensible format for the packaging and distribution of software to be run in virtual machines.”

An OVF descriptor not only can contain one virtual machine but multiple VMs, called systems of machines. According to the specification an OVF package can contain the following artifacts:

  • one OVF descriptor with extension .ovf
  • zero or one OVF manifest with extension .mf
  • zero or one OVF certificate with extension .cert
  • zero or more disk image files
  • zero or more additional resource files, such as ISO images

Furthermore it provides a descriptor file defining a virtual machines amount of memory and cpu as well as the network configuration. The manifest file contains SHA-1 digests of all files to ensure the integrity throughout multiple transfers. To ensure authenticity the manifest file can be signed and the digest stored in the certificate file. All artifacts can be compressed into an appliance (.ova as file ending) for easier distribution (as done here for Zimbra’s Collaboration Server). All major virtualization products support import and export of OVF files, so chances are, this standard will be around for a while..

Cloud Data Management Interface (CDMI)

Storage Networking Industry Association (SNIA) put some effort in specifying a Cloud Data Management Interface (CDMI). It “[…] defines the functional interface that applications will use to create, retrieve, update and delete data elements from the Cloud” (source). SNIA chose REST over HTTP for implementing the standard.

The standard defines a container and a data object as the major artifacts. Storage space is exposed as a container. Containers can be created with a parent child relationship to recursively group data objects , although this recursion is an optional feature. Following operations can be carried out on the afore-mentioned artifacts:

  • discovering the capabilities of a cloud storage provider
  • creating a new container
  • creating a new data object
  • listing the contents of a container
  • reading the contents of a data object
  • reading only the value of a data object
  • deleting a data object

The next code listing, taken from the specification, shows an example request for creating a container. One can see that the container should be reachable (or exports the data objects) through iSCSI as well as through NFS.

More detailed information on the given excerpt and examples as well as additional information on queues, domains, logging, snapshots and the concept of capabilities can be found in the specification.

Cloud Infrastructure Management Interface (CIMI)

The Cloud Infrastructure Management Interface is another standard defined by the DMTF. It includes a model for cloud infrastructure resources as well as a standardized REST over HTTP interface to manage said resources. Data is transferred either in XML or in JSON notation. Currently the model covers systems (cluster of machines), machines, volumes,  network and monitoring. CIMI is compatible with the OVF format which was mentioned earlier and also comes from the DMTF. OVF appliances can be imported (and exported) resulting in either the creation of templates or a concrete machine or system. The resources defined in the OVF appliance are translated to native CIMI resources.

Each CIMI implementation needs to provide a CloudEntryPoint to publish the supported resources and their corresponding  URIs for interaction. A CIMI client can discover all resources starting from the CloudEntryPoint, regardless of the implementation of the serving component.

The following example is taken from the specification and shows an update operation on properties of a machine. A new name is set and the description is being removed (because no value for description is provided).

The simplest version of a CIMI client is a HTTP client like curl.

This example is taken from the Deltacloud documentation and retrieves a list of existing machines. More examples can be found here.

Open Cloud Computing Interface (OCCI)

Much like CIMI the Open Cloud Computing Interface provides a REST over HTTP based way to create, update and destroy cloud infrastructure resources. It focuses on compute, network and storage only. Even the example of updating an instances attributes looks very similar (found in the specification):

OpenNebula for example already has built-in support for OCCI. From a quick look into the specifications of OCCI and CIMI I would say both are equally powerful IaaS standards.

Cloud Application Management for Platforms (CAMP)

CAMP tries to standardize the interaction with PaaS providers. The Organization for the Advancement of Structured Information Standards (OASIS) based project also uses REST for communication. The resources managed by the system are divided into platform, platform components,  applications components and assemblies. A platform describes a whole cloud offering. A platform component is a distinguished service offered by the platform. Application components are configured parts of an application. All application components (and resources) together form an assembly. Besides these resources CAMP also covers the whole lifecycle of an application as shown in the figure below.

CAMP Application Lifecycle

An application’s lifecycle (source http://docs.oasis-open.org/camp/camp-spec/v1.1/camp-spec-v1.1.pdf)

Similar to OVF, CAMP defines a Platform Deployment Package (PDP) to hand over an application’s data in a structured manner This package typically contains the following items:

  • one Deployment Plan file with extension .dp
  • zero or one manifest file with the extension .mf
  • zero or one certificate with the extension .cert
  • zero or more language specific bundles
  • zero or more additional resource files

Again, the manifest file is a list of files belonging to the PDP and can be signed through the certificate file. The deployment plan contains all the information on how to set up the environment needed by the application. It provides templates and configurations for the platform components and defines the needed dependencies.

Currently the standard only defines a database platform component (mainly because Oracle is the driving force?).

Open Services for Lifecycle Collaboration (OSLC)

Open Services for Lifecycle Collaboration (OSLC) is a framework of defined interfaces to integrate Application Lifecycle Management (ALM) and Product Life-cycle Management (PLM) tools. The specification of these interfaces is divided into different working groups that focus on a specific topic (automation for example). Through those interfaces the different tools can interact, share data and create a workflow that is spanning an application’s lifecycle with a common view for the user.

This is great news for users. No need to worry about integrating different products or finding workarounds for product incompatibilities. Also there is a huge opportunity for software companies. All products supporting a particular interface can be used to fulfill the task, no matter what other products it is supposed to integrate with. If you are interested, here is a short introduction to OSLC.

There is already a SDK to help you getting started with an OSLC implementation, too.

Last but not least the list of supporting organizations is very impressive.

Topology and Orchestration Specification for Cloud Services (TOSCA)

Topology and Orchestration Specification for Cloud Services (TOSCA) is a standard that provides a language framework to describe service templates. The primary target is to enable developers or architects to describe the topology of a service in an interchangeable format. Which software components does the service need and how do they interact? The strength of the approach lies in its generality. Fundamental artifacts are NodeTypes and RelationshipTypes. Both can be enriched with Properties and expose an Interface for management functionalities. A Service Template assembles  Node- and RelationshipTypes to a structured description of the cloud application or cloud service.

A NodeType can also be a recursive composite consisting of a Service Template wrapped in a Boundary Definition. This allows for the abstraction of complex applications. Node- and RelationshipTypes have to be backed by so called Implemetation Artifacts which contain the actual work in form of scripts or program code. A NodeType for Apache Tomcat Server for example could incorporate the operations Install, Uninstall, Start and Stop. A NodeType only defines the existence of those operations. Therefore it is also possible to have more than one implementation for a specific NodeType. TOSCA does not regulate which programming language has to be used for Implementation Artifacts. Rather it defines different runtime environments that a TOSCA-Container can provide (e.g. Python, Java, Shell). Also it is the developers decision wether he provides an implementation for all NodeTypes or if he defines a NodeType as abstract meaning that the TOSCA Container should use any matching implementation. This could be a valid scenario if one only needs a JEE application server but does not care which one is used.

More complex installation processes, which are not automatically derivable from the topology, can be described with Plans. All artifacts combined (definitions, resources, scripts, …) form the Cloud Service Archive (short CSAR). In the future a CSAR can be seen as WAR or EAR of the cloud to be exchanged between different cloud providers for deployment.

TOSCA Topology Template

Overview of TOSCA artifacts version 1.0 (http://docs.oasis-open.org/tosca/TOSCA/v1.0/cs01/TOSCA-v1.0-cs01.html)

Most important for the understanding of TOSCA is that it does not standardize a product or a certain topology for cloud applications but a language  for describing (cloud-) service topologies. With OpenTosca an open implementation of TOSCA was started by the University of Stuttgart. Most of the supporting product companies already have announced products for this standard. Usually these are not brand new products but additions to existing automation suites.

Chances are good that a whole new eco system will emerge around this standard, if it prevails. Like apps in an apps store, implementation artifacts could be made available and enable operations personnel to use the implementations that suit teir scenario best without implementing any product specific deployment logic. I’m really looking forward to see this happen!

Final Overview

Here is a short overview of the individual topics presented earlier and how they fit together.

 

Cloud standards sorted by topic

Cloud standards sorted by topic

As one can see, most implementations can be used simultaneously to pave your way to the cloud. TOSCA and OSLC are very high-level approaches to the cloud. Both can be combined as a TOSCA implementation can also support the corresponding OSLC interfaces (of automation for example).  Since TOSCA does not include pre-defined implementation artifacts product vendors need to provide them themselves or rely on 3rd party artifacts. Either way someone needs to fill implementation artifacts with operational logic. This effort can be minimized by leveraging OCCI or CIMI for infrastructure resources and CAMP for platform configurations.

APIs like libcloud or Deltacloud really help to lower the barrier for portable cloud deployments. They are a necessary steps in evolution towards an interoperable cloud but hopefully will be replaced by OCCI or CIMI one day.

Further Reading

This was only a short glance at cloud standardization. There are many standards [7, 8, 9] or tools [10, 11, 12] not covered here but also worth looking at.

 

 

Leave a Comment

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close