WikiPrint - from Polar Technologies

DETER Testbed API

The DETER testbed API is broken up into services, each covering one of the major abstractions that the testbed exports. The services are:

We discuss each in detail and provide links to the detailed javadoc of the implementation, where applicable and available.

The document and API are both works in progress and will be fleshed out and changed as the API is modified.

General Info

The API is implemented as a SOAP web service over TLS secured connections. The user's identity is generally provided by a client certificate, but the client certificates are generally transient. A valid temporary client certificate can be requested from the API by correctly responding to a challenge keyed on the user's password.

The API is implemented using the axis web service framework, which means that javadocs are provided for the various calls.

Each service and call are accessed by appending servicename/operation to the base URL of the API server. On DETERlab the base URL will be ​https://api.isi.deterlabnet:52323/axis2/service. For example, one can access the getVersion operation below at ​https://api.isi.deterlabnet:52323/axis2/service/ApiInfo/getVersion

Each operation returns useful parameters on success and throws a fault, called a DeterFault, on an error. Faults are standard SOAP faults with a detail section that includes the following fields:

The ErrorString and ErrorCode are equivalent, but the information in the DetailString is generally more informative about the specifics. Request or access errors are generally correctable on the client side while internal errors are not.

​javadoc for DeterFault

ApiInfo

The ApiInfo service provides metadata about the running DETER API. It also provides a simple check that a user is presenting authentication credentials correctly.

​javadoc for ApiInfo

The primary operation in the ApiInfo service is getVersion. The call is unauthenticated and can be made driectly from a web browser to confirm that the API is functioning and that the user can see DETERlab.

There is an addtional simpler service, echo, that takes one parameter and returns it:

Users

The Users API is concerned with managing users and their profiles as well as authenticating to the testbed and receiving a client certificate for later calls.

​javadoc for Users

Authentication

A user's identity is tied to an X.509 client certifcate issued by DETER. A certificate is bound to one user at a time, but a person can logout and log in as a different DETER user using the same certificate by presenting a different password, or logging out, which severs the connection of user to certificate.

A user can get a DETER-signed certificate by logging in without presenting an existing DETER-signed certificate. For example, the first time a user logs in, they will get a DETER-signed certificate.

A UI is welcome to keep one certificate for a session, or to reuse the certificates until they expire. The certificates are issued with several years of valid time.

A user can authenticate to the testbed and receive an X.509 certificate by requesting a challenge using the requestChallenge operation and responding to the challenge via the challengeResponse operation.

Currently only one challenge is available, type "clear". The challenge has no data and the user replies with their password in clear text. Note that the exchange is protected by the SSL encrypted exchange, but implementors should not respond to clear challenges with parameters in the query string. The "masked" challenge will address this issue shortly.

Each challenge is valid for 2 minutes and are rate-limited. Web interfaces should collect the password from the user before requesting a challenge from the API to avoid spurious timeouts.

Any certificate returned by the challenge is signed by the testbed.

The binding between user and certificate lasts 24 hours. It can be renewed by making another login exchange. A UI may want to prompt a user to login again before the binding times out.

Removes the user-to-certificate binding for this certificate. The certificate can be reused or discarded, but cannot be used to manipulate the testbed without another successful requestChallenge/challengeResponse exchange.

Password Changes

The password is a unique user feature that is not in the profile because of its role in authentication. When a user needs to change their password there are two API calls that can be used.

If the user knows their current (or expired) password, they can authenticate using the standard challenge response protocol and then call

We expect that the web interface will handle issues like confirming user input to a password change page. The changePassword call just makes the change directly.

If a user has forgotten their password, the user can request a password challenge, sent to them at their profile e-mail address. The challenge is a 64-bit number that can be used to call changePasswordChallenge without logging in. To request a challenge, the web interface calls:

Again, we expect this call to generally be made from a web interface that will then want to present an input form to the in order to reset their password. The urlPrefix field provides that hook. A web interface running on https://niftydeter.com might call requestPasswordReset with parameters 'forgetfuluser' and 'https://niftydeter.com/reset.html?challenge='. After that call forgetfuluser will get e-mail asking him or her to access the web page at https://niftydeter.com/reset.html?challenge=1283548127541824, allowing niftydeter.com to present their password change interface, and do error checking, etc. on the new password.

Each challenge is valid for 2 hours, and they are rate-limited so only a few can be outstanding.

With a valid challenge in hand, the web interface can call

Profile Manipulation

DETER keeps metadata about each user called a profile. The API provides an authenticated user with several interfaces to query and modify their profile information.

In the API each element of profile data is represented as a structure with the following data in it:

To get a profile schema, for example to create an empty web page,

To read a user's profile (generally only users can read their own profile).

Finally a user can modify a profile:

Notifications

A notification is a short message from another user or the testbed asking a user to take an action (add someone to a project, update a password before expiration) or communicating other information to the user (testbed downtime, etc). A web interface will want to present these to a user.

Each notification has a set of flags that define its state. Currently there are 2

The user is free to change this state using the markNotifications call. Note that notifications are per-user. A user who clears the READ or URGENT flag does not clear the flag for other recipients of the notification.

The filtering variables are all applied if given.

The mask and flags variable define the state a user is searching for. If a bit is set in both mask and flags, only messages with that bit set are returned. Similarly if a bit is clear in flags and set in mask, only messages with that bit clear will be returned.

To be concrete: a user who wants to retrieve all unread messages (urgent or not) should send a flags field with the READ bit clear and a mask with the READ bit set. To retrieve only unread urgent messages, the flags should have READ clear and URGENT set, and both READ and URGENT set in the mask.

Administrators can send notifications, for example to announce testbed events. The call to do so is:

Creation and Deletion

Finally a user can request access to the testbed by creating a profile. The user has no privileges and consumes minimal resources until they join a vetted project. This is an unauthenticated call, but the user profile is not created until an automated e-mail exchange is made between the testbed and the proto-user.

Note that all non-optional fields must be provided, so this is best preceeded by a call to getProfileDescription to learn the fields.

A base userid is constructed, either from the userid presented or from a required field in the profile. If that base userid is available, that a user with that userid is created. Otherwise the userid is disambiguated until there are no collisions. The userid actually created is returned.

The user is created without membership in any projects and without a password. A password reset challenge (equivalent to one issued by requestPasswordReset) is issued and the challenge e-mailed to the user at the address in the profile.

Administrators can create users without the confirmation step:

Note that either the cleartext password or the hash and type must be passed. This interface is under for initializing the testbed or creating users in batch for something like a class.

Removing a user is accomplished using

Most users are unlikely to use this service, and policy may allow only administrators to remove users.

Manipulating Profile Attributes

Administrators may add attributes to user profiles or remove them. That is to say, add a schema (including format constraints and descriptions), not set a value for a user.

Creating an attribute:

And removing an attribute:

Finally administrators can modify profile attributes:

Projects

The profile API lets users manage their project memberships as well as manipulate projects.

​Projects javadocs

Listing Projects

Authenticated users can look at the various projects they are members of, and may scope such searcher by project names.

Filters are cumulative: specifying a regular expression and an owner will match on both.

Changing Project Membership

Each of these changes propagates to the linked circle that controls resource permissions for the project. In particular, the permissions fields should include the union of the project and circle rights to set

With appropriate rights a user can add another user to a project they are in. The user being added must confirm this addition (below). As a result of this call, users being added will be sent a notification including a challenge id to respond to, similar to a password change URL. The prefix for that URL can be passed in here.

For each user successfully added, a notification is sent. To respond to that notification the UI must call:

Once the confirmation is received, the user is actually a member of the project. If challenges are not responded to in 48 hours, they are no longer valid.

Administrators can add users without going through the confirmation process:

Users can also request to join a project. This requires some member of the project with ADD_USER rights to agree, so a notification is sent to all such users.

The join succeeds when a user with appropriate permissions calls joinProjectConfirm. Note that the user calling joinProjectConfirm sets the new user's project permissions.

Users can be removed from projects:

If a user has the right to both add and remove a user, that user can change a user's permissions:

Manipulating Projects

The owner of a group (and only the owner of that group) can give that privilege away:

Project Profiles

The calls for manipulating a project profile are very similar to user profiles:

To get a profile schema, for example to create an empty web page,

To read a project's profile.

Finally a user can modify a project profile:

Manipulating Profile Attributes

Administrators may add attributes to project profiles or remove them. That is to say, add a schema (including format constraints and descriptions), not set a value for a project.

Creating an attribute:

And removing an attribute:

Finally administrators can modify profile attributes:

Creating and Deleting Projects

Creating and removing projects are also similar to creating and removing users:

As with users, all required (non-optional) fields in the profile must be provided. The project will be created, but have no rights until approved.

On success the new project is created with the calling user as owner and sole member. The user has all rights.

Approval is a result of the testbed's vetting process. The user will be sent a notification when their project is approved. When approved this call is made by an admin.

Note that this call can be used to un-approve a previously approved project.

Removing a project is straightforward

Generally an removing a project requires a testbed administrator.

Circles

The circle API lets users manage their circle memberships as well as manipulate circles. The circles API is similar to the Projects API. The primary exceptions are that there is no approval process for circles (though there is for joining or adding a user) and that the permissions are different.

These calls work on all circles except those linked to a project. Those circles must have their membership, ownership, and permissions manipulated through the project to ensure that the contents remain synchronized.

​circles javadoc

Listing Circles

Authenticated users can look at the various circles they are members of, and may scope such searcher by circle names.

Filters are cumulative: specifying a regular expression and an owner will match on both.

Changing Circle Membership

With appropriate rights a user can add another user to a circle they are in. The user being added must confirm this addition (below). As a result of this call, users being added will be sent a notification including a challenge id to respond to, similar to a password change URL. The prefix for that URL can be passed in here.

For each user successfully added, a notification is sent. To respond to that notification the UI must call:

Administrators can add users without going through the confirmation process:

Users can also request to join a circle. This requires some member of the circle with ADD_USER rights to agree, so a notification is sent to all such users.

The join succeeds when a user with appropriate permissions calls joinCircleConfirm. Note that the user calling joinCircleConfirm sets the new user's circle permissions.

Users can be removed from circles:

If a user has the right to both add and remove a user, that user can change a user's permissions:

Manipulating Circles

The owner of a group (and only the owner of that group) can give that privilege away to another member:

Circle Profiles

The calls for manipulating a circle profile are very similar to user profiles:

To get a profile schema, for example to create an empty web page,

To read a circle's profile.

Finally a user can modify a circle profile:

Manipulating Profile Attributes

Administrators may add attributes to circle profiles or remove them. That is to say, add a schema (including format constraints and descriptions), not set a value for a circle. Creating an attribute:

And removing an attribute:

Finally administrators can modify profile attributes:

Creating and Deleting Circles

Creating and removing circles are also similar to creating and removing users:

As with users, all required (non-optional) fields in the profile must be provided. The circle will be created, and can be used immediately.

Removing a circle is straightforward.

Experiments

The experiment interface controls managing experiment definitions and realizing experiments (allocating and initializing resources). The experiment definition is in some flux, which is reflected in this section.

An experiment is a collection of aspects, each of which controls part of the experiment's environment and operation. Some aspects are understood directly by the testbed, while others are interpreted and accessed by tolls inside the experiment. Such tools may be written by DETER administrators or by experimenters themselves.

An ​aspect is a named and typed object associated with the the experiment. Specifically, an aspect has the following attributes:

name
A unique string scoped by aspect type used to identify the aspect instance
type
A string that tells the testbed or tool how to interpret the aspect. The current interface interprets topology aspects as an experiment layout, and treats the rest as opaque.
subtype
A string characterizing the aspect further. Subtypes may be created by the testbed or tools.
data
The contents of the aspect. This will be empty if datareference has a value
datareference
A URI that points to the contents of this aspect
readOnly
A boolean, true if the aspect can be modified.

Each experiment also includes an access control list (ACL) that lists the permissions that members of a given circle are granted to this experiment.

​experiments javadoc

Viewing Experiments

One or more experiments can be viewed by an authorized user assuming that user has the proper permissions. The search can be scoped by library and a regular expression matched against the experiment name. In addition, the aspects returned and their contents can also be controlled. To view experiments, call:

Creating and Deleting Experiments

An experiment is created using:

Creating an experiment with one aspect may result in additional sub-aspects appearing, as well as modifications to other aspects.

An experiment is deleted using:

After removeExperiment succeeds, experiment state is removed from the testbed completely.

Modifying Experiments

Because aspects are somewhat free-form and can be interrelated, the interface for manipulating them is simple. Aspects can be added and removed.

As with creation, adding an aspect may result in additional sub-aspects appearing, as well as modifications to other aspects.

As with creation, adding an aspect may result in additional sub-aspects appearing, as well as modifications to other aspects.

The owner can give ownership to another user.

We expect to add filter-based interfaces in the future as well.

Realizing and Releasing Experiments

To realize an experiment use:

Experiment Profiles

Experiments have profiles attached to them, as users, circles, and projects do. Information in a profile is metadata about the experiment, intended for human consumption. This is in opposition to an aspect which is intended for the testbed or experiment infrastructure to interpret.

The calls for manipulating an experiment profile are very similar to other profiles:

To get a profile schema, for example to create an empty web page,

To read an experiment's profile.

Finally a user can modify an experiment profile:

Manipulating Profile Attributes

Administrators may add attributes to experiment profiles or remove them. That is to say, add a schema (including format constraints and descriptions), not set a value for an experiment.

Creating an attribute:

And removing an attribute:

Finally administrators can modify profile attributes:

Libraries

The library interface controls managing library contents. The experiment definition is in some flux, which is reflected in this section.

An library is a collection of experiments, each of which can be manipulated by users with the proper permissions. By grouping experiments it is easier to collaborate on the experiments as well as providing pools of experiments to use as starting points for new users.

Each library includes an access control list (ACL) that lists the permissions that mebmers of a given circle are granted to the library.

Viewing Libraries

One or more libraries can be viewed by an authorized user assuming that user has the proper permissions. The search can be a regular expression matched against the library name. To view libraries, call:

Creating and Deleting Libraries

A library is created using:

A library is deleted using:

After removeLibrary succeeds, library is removed from the testbed completely.

Modifying Libraries

Experiments can be added and removed from libraries

As with creation, adding an aspect may result in additional sub-aspects appearing, as well as modifications to other aspects.

The owner can give ownership to another user.

Library Profiles

Libraries have profiles attached to them, as users, circles, and projects do. Information in a profile is metadata about the library, intended for human consumption.

The calls for manipulating a library profile are very similar to other profiles:

To get a profile schema, for example to create an empty web page,

To read a library's profile.

Finally a user can modify a library profile:

Manipulating Profile Attributes

Administrators may add attributes to library profiles or remove them. That is to say, add a schema (including format constraints and descriptions), not set a value for a library.

Creating an attribute:

And removing an attribute:

Finally administrators can modify profile attributes:

Admin

The Admin service allows testbed administrators to manipulate the implementation internals and to initialize the testbed. Most users will use this interface. This section is provided out of order for completeness.

​javadocs for Admin

The various permissions described in this document are also installed in the implementation's database. It is possible to add new permissions while the system is running using the addPermission call. The name of a permission is a string (scoped by the object). Objects include circle, library, experiment, and project.

The bootstrap call initializes an empty system and adds a user with full administrative power. This only works if no admin project exists and no deterboss user exists. It also creates a regression project for testing. It returns the userid and password of the new admin user (always deterboss).

The clearCredentialCache() call removes any cached credentials from the system.

There are simple timers inserted in the code at times. One can access their values and reset them using:

If the policy databases have changed, administrators can reset the underlying policy and credential database using

This can be a time-consuming operation.

Resources

Resources are things that make up experiments. They are fairly generic at this point, though we expect to specialize them. Currently system resources - like computers and VMs - are primarily manipulated by the testbed. Like circles, resources have names scoped by user or project. A user can create local resources in their namespace and the testbed creates resources in the admin or system namespaces.

Users primarily see system resources when an experiment is realized.

A resource consists a type, an optional description, zero or more facets, zero or more tags and an ACL that allows circle members to manipulate it. Types are identifiers that control how the testbed and user can manipulate them. Common resource types include "VM" and "computer".

A resource encapsulates the ability to

Each of those capabilities is described by a facet data structure attached to the resource. A facet describes the available powers of a resource. When a resource is realized inside another, the inner resource provides its facets by consuming the outer resource's facets.

Tags are name value pairs associated with a resource or facet thereof. They are a simple extension mechanism, akin to attributes in topdl.

Viewing Resources

One or more resources can be viewed by an authorized user assuming that user has the proper permissions. The search can be a regular expression matched against the library name as well as scoping the search by type, realization membership, tags and user visibility. To view resources, call:

Users - primarily administrators - can create or delete resources

A resource is deleted using:

Realizations

A realization is a binding of an Experiment and a Circle to Resources. Intuitively, it is an experiment in progress. A researcher manipulates a realization using the Realizations service.

There are two bindings that are part of a realization, the containment hierarchy that shows how resources are bound to other resources and a topology mapping that maps topology elements to resources.

In addition a realization has a status, indicating the state of the experiment instance as a whole.

Viewing Realizations

One or more realizations - experiments in progress - can be viewed by an authorized user assuming that user has the proper permissions. The search can be scoped by circle, and a regular expression matched against the experiment name. To view realizations, call:

Creating and Deleting a Realization

A researcher creates a realization - starts an experiment - using the realizeExperiment operation on an experiment.

Removing A Realization

A realization is deleted using:

After removeRealization succeeds, the realization is removed from the testbed and the resources are released.

A user may deallocate the resources from a realization using:

A realization is deleted using:

After removeRealization succeeds, the realization remains in the testbed in the empty state. When future interface changes allow changes to realizations, this will be more useful.