Google Summer of Code '20 - Cloud Plugin

A blog post and starter material for the Cloud Plugin - work done during Google Summer of Code ‘20 on creating a CueCommander plugin that can be used to manage cloud pools from OpenCue.

Author : Shiva Kannan

Please feel free to reach out at opencue-user@lists.aswf.io if you have questions/feedback about this work.

Summer of code proposal

The main proposal consisted of the plan of action for the 3 months of development, which included research about finding a common language among cloud providers when it came to managing a pool of instances and the ability to create, delete and resize these cloud groups through their API from the OpenCue interface.

I took Google Cloud Platform as the primary cloud provider and tried to find parallels in other providers and started designing the main API around it.

A Design Document about the initial design was then shared with the community for feedback and review which included the specs of the API (which changed over the course of 3 months) and the API matrix which shows what the parallels in each provider are.

Cloud plugin source code

The code base for this plugin is currently on the cloud-plugin branch of the OpenCue repo.

Design overview

API design overview

Most cloud providers have the ability to manage a group of instances that can be created using a VM template/source VM and can be managed as a single entity instead of managing separate instances. They go by different names in different providers: Instance Groups (GCP), Scale Sets (Azure), Auto Scaling Groups (AWS).

The design takes into consideration how each of the cloud provider’s API handle creation/scaling/deletion of these entities and how they will be used by the plugin’s single interface to manage multiple cloud provider’s entities.

The design consists of two main abstract classes:

1. CloudManager

   CloudManager class is the interface between the API and the widget. The CloudManager is also an abstract class that will be implemented by each registered provider. This abstract class handles the following:

   • connect(): Connection mechanism to the cloud provider through its corresponding API.

   • create_managed_group(): Creation of a cloud pool group (CloudInstanceGroup in the API terminology).

   • get_all_groups(): Lists all the cloud pool groups that are available in the cloud provider’s project.

   Google Cloud Platform implementation of this class can be found here : GoogleCloudManager

2. CloudInstanceGroup

   The CloudInstanceGroup abstract class is the container that handles the cloud provider’s managed group entity. This takes care of the following:

   • Handles setting important attributes like name, id, group’s instances and the status of the cloud group.

   • resize(): Resizing the group (scaling up/down). Currently it’s left to the cloud provider to take care of which instances to remove during the process of scaling down.

   • delete_cloud_group(): Deletion of the cloud group from the cloud provider.

   Google Cloud Platform implementation of this abstract class can be found here: GoogleCloudGroup

Plugin widget design

The CloudManager is a plugin for CueCommander. There are three levels to implementing this entire widget:

1. Creating the widget inside of the plugins directory. The module level variables to be defined for the plugin are mentioned in the docs of the plugin registering script. This helps registering the plugin, making it show up in the submenu and making it launch-able through the menu item.

2. Implementing AbstractDockWidget for the CloudManagerDockWidget to create a dockable widget inside of the CueCommander interface.

3. The plugin itself then puts together two separate widgets:

   • CloudManagerTreeWidget: This implements the AbstractTreeWidget class which provides very useful methods to handle the way data has to be displayed in the widget.

   • CloudGroupCreateDialog: Which is used outside of the tree view and launched when the Add Cloud Group button is clicked on the plugin.

The interface uses the main CloudManager api, mentioned above, as the entry point to then get all cloud groups associated with all the registered cloud providers.

Cloud plugin setup

Google compute engine setup

Listing the minimum requirements for the cloud plugin to work with GCE:

1. For your OpenCue deployment, make sure that the GCE nodes are able to reach your Cuebot on port 8443 and for Cuebot to talk to RQD, make sure Cuebot is able to reach port 8444 on the nodes. In my approach, I deployed Cuebot on GCE itself and the configuration was relatively simple. If you are running Cuebot locally, either configure the firewall rules in both directions or have a VPN connection to GCP.

2. GCP project with Compute Engine API and Cloud Logging API enabled.

3. Follow Application Default Credentials to setup a easy credentials access for the google APIs that the plugin uses.

4. Once you setup Application Default Credentials, the APIs should pick it up automatically for authentication. Please refer to passing credentials manually if otherwise.

5. Modify the cloud_plugin_resources.yaml under the cuegui/config directory to setup your GCP project name and the zone name that the API will use to access the resources from.

6. At least one instance template setup on the cloud: I was using a RQD Blender docker image uploaded as an Instance Template with its appropriate VM type. Few things to note here regarding Instance Templates:

   • The docker image I used launched RQD using the container’s startup command so I didn’t have to explicitly specify any startup script.

   • If you are using a custom setup/running GCE instances without containers, make sure the startup script for the VM launches RQD when the VM is provisioned.

   • The CUEBOT_HOSTNAME environment variable must be set for RQD to point to Cuebot. In my deployment, this env variable was set to the internal IP address of Cuebot running on GCE.

Cloud plugin interface

The Cloud plugin should be available to launch from the menu bar by navigating: Views/Plugins -> CueCommander -> Cloud Plugin

1. The cloud plugin interface consists mainly of two components:
   1. Main window which shows the list of cloud groups available with all registered providers.
   2. A cloud group addition dialog that can be used to create a cloud group from the interface itself.

Cloud Plugin functionality

1. To start create a cloud pool group, use the Add Cloud Group button to launch the dialog that will allow you to create a cloud group.

2. If the API request is successful you should now see an entry in the Cloud Plugin’s main interface.

3. The right-click content menu has two additional functionality:
   1. Resizing (scaling up/down) the number of instances in a group.
   2. Deleting the group altogether.

4. Once these cloud instances startup and RQD launches, you should see these nodes appear in the Monitor Hosts plugin, after which OpenCue will be able to send jobs to these hosts.

Further work and updates

1. An Azure plugin that uses the same interface is currently a work in progress. A draft PR for the same can be found here: Azure Plugin PR.

2. Local operation caching.

3. AWS integration for the cloud plugin.

4. Better integration with the MonitorHosts plugin: Cloud group source for MonitorHosts plugin.

Please feel free to contact opencue-user@lists.aswf.io if you have any questions/feedback about this work.

• ← Previous
Next →