DETER Testbed API

The DETER API is really an interface with two sides, an outward facing side (the testbed API) that allows people to manage resources on the testbed and an inward facing side (the containers API) that coordinates the containers that make up an experiment environment. This document defines these interfaces and documents how to use them. It is a living document that will gain detail as the specification grows and implementations are put into service. At this point (July 2013) it is mostly a roadmap.

The testbed API is responsible for managing the following things:

Users

researchers

Projects

groups of related users, used to manage what users can see and do to other testbed resources

Resources

building blocks for realizing experiments: computers, disk images, external access

Experiments

a research environment that may be stored, edited, and realized on testbed resources

The testbed API how users ask a DETER testbed to do things for it. It will generally be called from a more user-friendly front end, tuned to the user's experience and goals, such as the evoloving DETER Beginners Interface.

The containers API is used by the DETER control system to manage the resources that make up an experimental topology in progress. The goal of this API is to take raw resources inside the testbed that have been allocated using the testbed API, and configure them into a usable environment. Each logical element of an experiment (a computer, a router, etc.) is represented by a container which must be managed by DETER.

Managing a container consists of:

• Installing and configuring any virtualization software or configuring hardware
  • This involves translating from a more generic topology/configuration description into the setup for a specific container type
• Configuring and starting MAGI software necessary to connect to the experiment management system
• Exposing and using container-specific features to provide DETER services

We discuss the testbed API and then the containers API. Then we discuss some initial implementation details and present a roadmap for development.

DETER Testbed API

The testbed API manipulates a few entities to provide the experimentation environment. We begin with a section detailing those entities conceptually and then discuss the specific actions that the API allows on them.

Testbed Entities

Users

A user is a researcher who uses the DETER testbed. They request testbed services and allocate testbed resources. Users are the actors that make things happen through test testbed API.

Each user is identified by a unique string, their userid. Userids are assigned when the user is created on the testbed and guaranteed to be unique.

In addition to the user identifier DETER keeps meta-data about users. Currently that meta-data consists of:

• Projects the user is in (see below)
• Experiments the user owns (see below)
• A password to authenticate the user
• A valid e-mail address for communication and password resetting
• Resource access information, e.g.,
  • ssh public keys
  • windows authentication credentials
• General metadata, e.g,
  • Real name
  • Affiliation
  • Phone number
  • Address

A user identifies themselves to the testbed API by proving that they hold a specific public/private keypair. An initial such keypair is issued when the user is created, and a user can acquire another valid pair at any time by proving they know their password. Generally those pairs are short-lived to guard against loss or theft, but the password is administered using local testbed policies.

Projects

Projects are groups of users. They are used to confer rights to groups of users, identify groups of users, and control how resources are configured when experiments are realized on the testbed. This is a more general use of projects than an Emulab testbed.

A project always has an owner, the user responsible for its creation. Ownership can be changed, but only by the owner.

When a user is added to a project, their rights are also defined within that project. The rights are:

• Can add other users or projects to the project
• Can remove users or projects from the project
• Can realize experiments under this project (granting members of the project access to the resources in use)

When a project is added to another project, the rights of the project are given to each member of the group.

Projects can also have attributes attached to them. The most important of these is the vetting attribute. A vetting project (one with the vetting attirbute) can allow users to have access to testbed resources. Until a user is a member of a vetting project, minimal testbed resources are allocated to them and they cannot perform any useful actions, except attempting to create one vetting project. DETER testbeds can be configured to alert staff when a vetting project is requested and require authorization from an administrator.

Vetting projects solve the problem of distributed user management on a large testbed. DETERlab has thousands of users and a small staff. The staff needs to make sure that users meet certain criteria. Having the staff review each user application is unreasonable, reviewing the creation of a comparatively smaller set of vetting projects is not.

When a researcher wants to start a project on DETERlab, the researcher creates a user and requests a vetting project. The vetting project is reviewed, and if valid created with the user as owner. The user is then added to the vetting project with the right to add others. Now that user is responsible for managing new people in their project. A new person creates a user that has no power until a researcher with a vetting account adds them to that project.

Though the testbed API sees this as two steps, the user interface presented by a web page would roll these two steps together. A user sees it as a single application page.

A testbed with looser constraints on the vetting process can skip or remove the review process.

If a user is removed from all vetting projects, they return to a powerless state.

Resources

Resources are the physical and conceptual objects managed by the testbed that are used to build experimental environments. They are the computers, network ports, externally routable addresses, virtual machine images, et al. from which experiments are constructed.

Access to and configuration of resources is affected by the project(s) a user is acting as a member of. When a user requests resources, thay specify the projects under which they are requesting them. A user requesting resources as a member of a project representing a university class may have access to different resources than one acting as a member of the testbed administration. How membership affects the resources a group can claim is set by testbed policy?.

In addition, the membership in projects controls how resources will be configured. A resource in use by a particular project will generally be configured to be accessible to all members of that project. A student who allocates resources as a member of a small design group while implementing a class project may later allocate resources as a member of the whole class when presenting the work to the class's TA and professor.

The API provides an authenticated user (see below) with several interfaces to query and modify their profile information.

In the API each element of profile data is represented as a structure with the following data in it:

• name of the element
• type of the element
  • string
  • integer
  • double
  • binary/opaque
  • lists of any of the above
• value(s) of the element
• a flag set if the element is optional
• a modification type: elements may be read/write, read-only (e.g., username) or write-only (e.g., password)
• a brief description of the field, intended to be presented by a web interface or other third party program

The interfaces to manipulate a profile are

• GetProfileDescription
  • No input required
  • Returns a set of empty elements that this testbed understands and requires, represented as above
• GetUserProfile
  • No input required execpt an authenticated user
  • Returns a populated set of elements containing the current values of the user's profile
• SetUserProfile
  • A list of populated elements that the (authenticated) user wishes to change in the current profile
  • A list of the elements successfully updated, and a list of those that failed with a reason for each

Authenticating as a User

A user represnts themself to the testbed by presenting proof that they hold a public key, for example through an SSL connection. The testbed determines what operations a user can carry out based on an ABAC-encoded policy that grants rights to users and binds keys to users. A user is issued a public key periodically by DETER, but also can request a short lived temporary key at any time by authenticating against a shared password. A temporary key is useful to allow a web application or other hosted tool to act on a user's behalf.