The DETER System Programming Interface (SPI) is a standard interface for manipulating fundamental DETER abstractions used to manage testbed resources and to marshall those resources into network and cybersecurity experiments. The SPI is analogous to an operating system's system call interface. It provides the fundamental abstractions that enable application programmers to make tools that are useful to researchers.
There are two motivations for building the SPI: first that the SPI is a context and programming framework for new DETER abstractions that subsumes the existing Emulab codebase cleanly. Second the SPI provides a context in which new developers can extend those abstractions to provide new functionality and to improve their scaling properties.
The SPI is fundamentally a tool-builder's interface. Researchers performing experiments or students performing class exercises are unlikely to access SPI methods directly, though we expect that higher level tools and interfaces will make the fundamental abstractions visible.
There are 6 first class abstractions that make up the DETER SPI. These are:
We describe each of these in more detail below.
This section motivates and explains two of the implementation design decisions that underlie the SPI: expanding the implementation beyond the Emulab abstractions and implementation, and using SOAP and web services to deliver the service.
The DETER research agenda drives the new abstractions, and it differs somewhat from the abstractions that drive the Emulab-derived code base. DETER is interested in sharing experiment infrastructure - including conducting experiments where multiple parties manipulate the environment based on different views of the world and different goals (multi-party experiments). Also, DETER is interested in multi-fidelity, large-scale experiments where the representation of the various partes of the experiment is guided by the researcher's design of the experiment. Finally, we think of experiments as having aspects that go well beyond the definition of the topology or layout, and the abstractions need to support this.
The Emulab code is based on simpler topology and experimentation models, and while its developers have been able to extend it to do more, our changes take the abstractions far enough that relegating the Emulab code to allocating local resources seems the cleanest way forward. Functions including user and project management will move into the SPI, though subsets of that function (basically current Emulab functionality) will remain available through the legacy interface.
As a last practical point Emulab-based codebase blurs the lines of some implementations between the webserver interface and the backend code. Building the SPI offers DETER a chance to solidify that interface and enable developer to build multiple applications on the DETER abstractions.
The SPI is built on the SOAP web services platform as a compromise between accessibility, extensibility, and support from tools. An important goal of the SPI is to allow multiple applications to use the DETER testbed. A web front-end for teaching cybersecurity classes that coexists with a web front end for experimenting on Internet anonymity are such a pair of applications. That may even look like two different testbeds from the outside.
To support that, the SPI must be platform independent. A web browser is a powerful application platform that the SPI must support. The SOAP/WSDL standards are a proven technology for such access. SOAP may be somewhat stodgier than a JSON/REST interface, but the differences can be constrained to encodings, and many tools for generating SOAP interfaces provide migration paths to other web services platforms.
Presently we take advantage of the fairly rich SOAP toolspace and have been able to use SOAP for access from a range of platforms from web browsers to multiple standalone programming languages.
The rest of this document goes into more detail on how to use the SPI.
This section describes the SPI abstractions and operations that application writers will use. We are concerned with how to put the concepts into use and how they interrelate with each other. Below we describe the detailed interface.
A profile is a collection of metadata attached to an abstraction instance, primarily to help users and administrators understand what the instance is bing used for. Users, projects, circles, experiments, and libraries all have profiles attached to them which describe those things. For example, a user profile includes the researcher's name and an e-mail address.
Profiles are self-describing, so that applications have guidance is what is being presented and how to display it. Each of the abstractions above has an operation called getProfileDescription that returns an array of all the valid fields - called attributes - in that abstraction's profile. Each element of that array - each attribute - includes additional metadata that describes its purpose and presentation:
| Metadata | Meaning |
|---|---|
| Name | Descriptive name for the attribute, e.g. e-mail. All attributes have a name. |
| Value | The contents of the attribute. When retrieving a profile description, every attribute's value is empty |
| Description | Brief natural language description of the attribute, suitable for context sensitive help |
| Access | Whether a user can modify the data. Valid values are READ_ONLY, READ_WRITE, NO_ACCESS, and WRITE_ONLY
|
| Optional | True if the field can be omitted in a profile. All non-optional fields must be present when creating an instance. Applications may expect these fields to be present. |
| DataType | How to interpret the data. Valid values are STRING, Int, FLOAT, and OPAQUE.
|
| Format | A regular expression describing valid field entries. Applications can use this to check user input locally. Applications are not required to do so, but values for this attribute that do not match the expression will be rejected. |
| FormatDescription | A natural language description of valid input suitable for context sensitive help. The FormatDescription for a phone number field might be "10 digits, optionally separated by hyphens or parentheses." |
| LengthHint | A hint at how much space an application should to reserve for input, in characters. Applications can obviously ignore this. |
| OrderingHint | An integer that suggests to an application what order to display the field relative to others. Applications can ignore this |
Each instance has an associated operation, getUserProfile (projects have getProjectProfile, experiments have getExperimentProfile) that returns the profile for that instance as an array of attributes, as above. This profile includes the values for all the (readable) attributes.
Each instance supports a changeUserAttribute operation that allows the application to change the instance's attribute values. changeUserAttribute is used to change a user's phone number. Optional attributes can be deleted through this interface.
An application will use the profile operations in the following ways:
getProfileDescription, fill in non-optional fields (probably based on user input) and include that profile data with the creation request. All abstractions that have profiles attached require a valid profile as a parameter to the operation that creates them.
getUserProfile et al., and display all or some of the profile data.
changeUserAttribute, et al.