wiki:NewTestbedAPISpec

Context Navigation

Version 5 (modified by faber, 11 years ago) (diff)
--

DETER Testbed API

The DETER testbed API is broken up into 5 services, each covering one of the major abstractions that the testbed exports. The services are:

ApiInfo
Users
Projects
Resrources
Experiments

We discuss each in detail and provide links to the detailed javadoc of the implementation, where applicable and available.

The document and API are both works in progress and will be fleshed out and changed as the API is modified.

General Info

The API is implemented as a SOAP web service over TLS secured connections. The user's identity is generally provided by a client certificate, but the client certificates are generally transient. A valid temporary client certificate can be requested from the API by correctly responding to a challenge keyed on the user's password.

The API is implemented using the axis web service framework, which means that javadocs are provided for the various calls.

Each service and call are accessed by appending servicename/operation to the base URL of the API server. On DETERlab the base URL will be https://api.isi.deterlabnet:52323/axis2/service. For example, one can access the getVersion operation below at https://api.isi.deterlabnet:52323/axis2/service/ApiInfo/getVersion

Each operation returns useful parameters on success and throws a fault, called a DeterFault, on an error. Faults are standard SOAP faults with a detail section that includes the following fields:

ErrorCode - a 32-bit integer encoding the type of error. Constants are available in the javadoc for DeterFault. Values are:
- access - access denied
- request - bad request
- internal - internal server error
ErrorString - a string describing the broad error
DetailString - a string describing the details that caused the error

The ErrorString and ErrorCode are equivalent, but the information in the DetailString is generally more informative about the specifics. Request or access errors are generally correctable on the client side while internal errors are not.

javadoc for DeterFault

ApiInfo

The ApiInfo service provides metadata about the running DETER API. It also provides a simple check that a user is presenting authentication credentials correctly.

javadoc for ApiInfo

The sole operation in the ApiInfo service is getVersion. The call is unauthenticated and can be made driectly from a web browser to confirm that the API is functioning and that the user can see DETERlab.

Service: ApiInfo
Operation: getVersion
Input Parameters:
- None
Return Values:
- Version - A string containing the API version number
- PatchLevel - A string containing the patch level
- KeyID - A string. If the user presented a valid public key and passed the challenge, this is the sha1 hash of that key. If no key was presented or an invalid one, this filed is not returned.

Users

The Users API is concerned with managing users and their profiles as well as authenticating to the testbed and receiving a client certificate for later calls.

javadoc for Users

Authentication

A user can authenticate to the testbed and receive a short-lived client x.509 certificate by requesting a challenge using the requestChallenge operation and responding to the challenge via the challengeResponse operation.

Currently only one challenge type is defined. The challenge is an array of bytes and the response is return an SHA1 HMAC of that string keyed by the user's password.

Service: Users
Operation: requestChallenge
Input Parameters:
- userid - the identity to authenticate
Return Values:
- Type - the kind of challenge
- Data - the data needed for the challenge
- Validity - number of seconds the challenge may be responded to
- ChallengeID - a 64-bit identifier that allows the server to validate the reply

Service: Users
Operation: challengeResponse
Input Parameters:
- ResponseData - a binary string, the response to the challenge
- ChallengeID - The 64-bit identifier of the challenge being responded to
Return Values:
- Certificate - a binary string containing a PEM-encoded X.509 certificate and private key. (Passed down the encrypted connection)

Profile Manipulation

DETER keeps metadata about each user called a profile. The API provides an authenticated user with several interfaces to query and modify their profile information.

In the API each element of profile data is represented as a structure with the following data in it:

name of the element
type of the element
- string
- integer
- double
- binary/opaque
value(s) of the element
a flag set if the element is optional
A flag set if the field can be removed from the profile
a modification type: elements may be read/write, read-only (e.g., username) or write-only (e.g., password)
a brief description of the field, intended to be presented by a web interface or other third party program

To get a profile schema, for example to create an empty web page,

Service: Users
Operation: getProfileDescription
Input Parameters:
Return Values:
- Uid - always empty
- A list of profile elements each containing
  - Name - a string, the element's name
  - DataType - a string giving the element's
    - string
    - integer
    - double
    - binary/opaque
  - StringValue - a string containing the element's value, unless it is binary/opaque
  - BinaryValue - a byte string containing the element's value if it is binary/opaque
  - Access - a 32-bit integer describing the access values (values at Attribute's javadoc)
    - READ_WRITE
    - READ_ONLY
    - WRITE_ONLY (e.g., password)
  - Optional - a flag true if the field is optional (must be present but may be empty)
  - Removable - a flag true if the field can be removed
  - Description - a string explaining the field

To read a user's profile (generally only users can read their own profile).

Service: Users
Operation: getProfileDescription
Input Parameters:
- userid - a string naming the user to be retrieved
Return Values:
- Userid - the user whose profile is returned
- A list of profile elements each containing
  - Name - a string, the element's name
  - DataType - a string giving the element's
    - string
    - integer
    - double
    - binary/opaque
  - StringValue - a string containing the element's value, unless it is binary/opaque
  - BinaryValue - a byte string containing the element's value if it is binary/opaque
  - Access - a 32-bit integer describing the access values (values at Attribute's javadoc)
    - READ_WRITE
    - READ_ONLY
    - WRITE_ONLY (e.g., password)
  - Optional - a flag true if the field is optional (must be present but may be empty)
  - Removable - a flag true if the field can be removed
  - Description - a string explaining the field

Finally a user can modify a profile:

Service: Users
Operation: getProfileDescription
Input Parameters:
- Userid - the user's profile to modify
- A list of change requests. Each request contains
  - Name - the name of the field to change
  - StringValue - the new value of the field if this is not an opaque/binary field
  - BinaryValue - the new value of the field if this is an opaque/binary field
  - Delete - a flag, if true delete the field rather than modify it
Return Values:
- A list of responses each containing
  - Name - astring with the name of the field
  - Success - a flag indicating if the request succeeded
  - Reason - a string indicating the reason if Success is false

Notifications

A notification is a short message from another user or the testbed asking a user to take an action (add someone to a project, update a password before expiration) or communicating other information to the user (testbed downtime, etc). A web interface will want to present these to a user.

Service: Users
Operation: getNotifications
Input Parameters:
- Userid - the user's identity to gather notifications
- FirstDate - an optional date. If present only show notifications after that date
- LastDate - an optional date. If present only show notifications before that date
- OnlyUnread - a flag. If true, show only unread notifications
Return Values:
- A list of responses each containing
  - ID - a 64-bit integer identifying the notification
  - IsRead - a flag indicating that the message has been read by this user
  - Sent - the date the notification was issued
  - Text - the text of the notification

The filtering variables are all applied if given. If OnlyUnread is true and FirstDate and LastDate are both given, only unread notifications between those dates are returned.

Service: Users
Operation: markNotificationsRead
Input Parameters:
- Userid - the user's identity to gather notifications
- Notifications - an array of 64-bit integers, the notifications to mark read
Return Values:
- None

Creation

Finally a user can request access to the testbed by creating a profile. The user has no privileges and consumes minimal resources until they join a vetted project. This is an unauthenticated call, but the user profile is not created until an automated e-mail exchange is made between the testbed and the proto-user.

Service: Users
Operation: getProfileDescription
Input Parameters:
- Userid - the requested userid
- A list of profile elements each containing
  - Name - a string, the element's name
  - StringValue - a string containing the element's value, unless it is binary/opaque
Return Values:
- None

Note that all non-optional fields must be provided, so this is best preceeded by a call to getProfileDescription to learn the fields.

Projects

The profile API lets users manage their project memberships as well as manipulate projects.

Listing Projects

Authenticated users can look at the various projects they are members of, and may scope such searcher by project names.

Service: Projects
Operation: viewProjects
Input Parameters:
- Userid - the requester's userid
- Owner - a userid - only find projects owned by this user
- NameRE - a string containing a regular expression to match against project names
Return Values:
- A list of project descriptors containing
  - Name - a string contaiing the project name
  - Members - an array of structures containing
    - Userid - a member's userid
    - rights - a 32-bit integer encoding the user's rights (each a flag)
      - ADD_USER - right to add a user to this group
      - REMOVE_USER - right to remove a user from this group
      - REALIZE_EXPERIMENT - the right to realize an experiment under this group
  - Vetting - a flag indicating this is a vetting project

Filters are cumulative: specifying a regular expression and an owner will match on both.

Manipulating Projects

With appropriate rights a user can add another user to a project they are in:

Service: Projects
Operation: addUser
Input Parameters:
- Userid - the requester's userid
- ProjectName - the project to change
- NewUser - the uid to add to the group
Return Values:
- None

If this fails, a fault is sent. A call to viewProjects will confirm the change.

Similarly, a user can be removed:

Service: Projects
Operation: removeUser
Input Parameters:
- Userid - the requester's userid
- ProjectName - the project to change
- RemoveUser - the uid to remove from the group
Return Values:
- None

If a user has the right to both add and remove a user, that user can change a user's permissions:

Service: Projects
Operation: changePermissions
Input Parameters:
- Userid - the requester's userid
- ProjectName - the project to change
- TargetUser - the uid to manipulate
- Rights - a 32-bit integer encoding the user's rights (each a flag)
  - ADD_USER - right to add a user to this group
  - REMOVE_USER - right to remove a user from this group
  - REALIZE_EXPERIMENT - the right to realize an experiment under this group
Return Values:
- None

The owner of a group (and only the owner of that group) can give that privilege away:

Service: Projects
Operation: setOwner
Input Parameters:
- Userid - the requester's userid
- ProjectName - the project to change
- NewOwner - the uid of the new owner
Return Values:
- None

A user may request to join an existing project. Note that the user need not be able to view or manipulate the project to make a request. The request will send a notification to users in the target group that can add users.

Service: Projects
Operation: requestJoin
Input Parameters:
- Userid - the requester's userid
- ProjectName - the project to join
Return Values:
- None

Creating and Deleting Projects

Non-vetting projects can be created and deleted with little overhead:

Service: Projects
Operation: createProject
Input Parameters:
- Userid - the requester's userid
- Name - the new project's name
Return Values:
- None

On success the new project is created with the calling user as owner and sole member. The user has all rights.

Service: Projects
Operation: removeProject
Input Parameters:
- Userid - the requester's userid
- Name - the new project's name
Return Values:
- None

Generally an owner can remove a project, but removing a vetting project requires a testbed administrator.

Because creating a vetting project may require review, it is more involved. First the user retrieves the testbed's schema for information to supply:

Service: Projects
Operation: getVettingRequirements
Input Parameters:
Return Values:
- A list of requirements elements each containing
  - Name - a string, the element's name
  - DataType - a string giving the element's
    - string
    - integer
    - double
    - binary/opaque
  - StringValue - a string containing the element's value, unless it is binary/opaque
  - BinaryValue - a byte string containing the element's value if it is binary/opaque
  - Optional - a flag true if the field is optional (must be present but may be empty)
  - Description - a string explaining the field

A web interface will gather this interface and call:

Service: Projects
Operation: createVettingProject
Input Parameters:
- Userid - the requester's userid
- Name - a string containing the project name
- A (possibly empty) list of vetting info fields requests. Each request contains
  - Name - the name of the field supplied
  - StringValue - the value of the field if this is not an opaque/binary field
  - BinaryValue - the value of the field if this is an opaque/binary field
Return Values:
- None

When this succeeds the data will be passed into the testbed's vetting process. The user will be notified out of band if the project was created. Behind the scenes the project is created by an admin using createProject and this call is made on it by the admin:

Service: Projects
Operation: makeProjectVetting
Input Parameters:
- Userid - the requester's userid
- Name - the project's name
Return Values:
- None

And then the admin adds the requesting user to the project makes them owner and removes themselves.

Experiments

The experiment interface controls managing experiment definitions and realizing experimnets (allocating and initializing resources). The experimnet definition is in some flux, which is reflected in this section.

Viewing Experiments

One or more experiments can be viewed by an authorized user assuming that user has the proper permissions. The creator of a project owns it and that userid scopes its name. That is an experiment's name is a pair: (owner, name). This allows experiments to be instantiated in different projects.

Service: Experiments
Operation: viewExperiments
Input Parameters:
- Userid - the user making the request
- ExperimentRE - a string containing a regular expression matched against project names (may be omitted)
- Owner - a string holding the owner of experiments to find (may be omitted)
- Components - a 32-bit integer containing a mask of fields to include
  - EXPERIMENT_TOPOLOGY - include the topology
  - EXPERIMENT_ACTIONS - include the actions
  - EXPERIMENT_CONSTRAINTS - include constraints
  - EXPERIMENT_DATACOLLECTION - include data collection
  - EXPERIMENT_RESOURCES - include resources needed or allocated
  - EXPERIMENT_CONTAINERS - include components if realized
  - EXPERIMENT_LOG - include realization log and fault
  - EXPERIMENT_PERMISSIONS - include project permissions
- TopologyFormat - an optional 32-bit int naming the topology format
  - TOPOLOGY_TOPDL - return a topdl file
  - TOPOLOGY_NS2 - return an ns2 file
- TopologyFilter - an optional filter for limiting scope of the topology file format TBD
- ActionFormat - an optional 32-bit int naming the action file format (values TBD)
- ActionFilter - an optional filter for limiting scope of the constraint file format TBD
- ConstraintFormat - an optional 32-bit int naming the constraint format (values TBD)
- ConstraintFilter - an optional filter for limiting scope of the constraint file format TBD
- DataCollectionFormat - an optional 32-bit int naming the data collection format (values TBD)
- DataCollectionFilter - an optional filter for limiting scope of the data collection file format TBD
- ResourceFilter - an optional filter for limiting scope of the resouces returned format TBD
- ContainerFilter - an optional filter for limiting scope of the constainers returned format TBD

Return Values:
- One or more structures with the following fields
  - Name - a string containing the experiment name
  - Owner - a string containing the owner's userid
  - Project - an optional string. If present the experiment is realized under this project
  - Topology - an optional file describing the experiment topology subject to filters and format constraints
  - Actions - an optional file describing the experiment actions subject to filters and format constraints
  - Constraints - an optional file describing the experiment constraints subject to filters and format constraints
  - DataCollection - an optional file describing the experiment data collection subject to filters and format constraints
  - Containers - an optional list with the following fields describing the containers in the experimnet, if it is realized. Subject to filtering.
    - Name - the container name
    - State - a string indicating state (Up, Down, Configured, Pinned, None)
    - Type - a string indicating the container type
  - Resources - an optional list of structures giving resources needed by or allocated to the experiment, subject to filtering.
    - Name - resource name
    - Type - resource type
  - State - a string, one of
    - "Unrealized", "Realized", "Changing", "Pinned" (Pinned means that some modification other than realization is underway)
  - StateDetail - a string giving additional state information. values TBD
  - Log - an optional file containing messages from the last realization attempt for debugging
  - FaultInfo - an optional structure containing the high level error message (if any) from the last realization. It contains the error fields from a DeterFault:
    - ErrorCode - a 32-bit integer encoding the type of error. Constants are available in the javadoc for DeterFault. Values are:
      - access - access denied
      - request - bad request
      - internal - internal server error
    - ErrorString - a string describing the broad error
    - DetailString - a string describing the details that caused the error
  - ReadProjects - a list of project names allowed to read this experiment
  - WriteProjects - a list of project names allowed to modify this experiment
  - RealizeProjects - a list of project names allowed to realize this experiment

Creating and Deleting Experiments

An experiment is created using:

Service: Experiments
Operation: createExperiment
Input Parameters:
- Userid - the user making the request (the owner on success)
- Name - a string containing a new experiment's name
- Topology - an optional file describing the experiment topology
- Actions - an optional file describing the experiment actions
- Constraints - an optional file describing the experiment constraints s
- DataCollection - an optional file describing the experiment data collection
- ReadProjects - a list of project names allowed to read this experiment
- WriteProjects - a list of project names allowed to modify this experiment
- RealizeProjects - a list of project names allowed to realize this experiment
Return Values:
- None

An experiment is modified using:

Service: Experiments
Operation: modifyExperiment
Input Parameters:
- Userid - the user making the request (the owner on success)
- Owner - the experiment owner
- Name - a string containing the experiment's name
- Topology - an optional file describing the experiment topology
- Actions - an optional file describing the experiment actions
- Constraints - an optional file describing the experiment constraints s
- DataCollection - an optional file describing the experiment data collection
- ReadProjects - a list of project names allowed to read this experiment
- WriteProjects - a list of project names allowed to modify this experiment
- RealizeProjects - a list of project names allowed to realize this experiment
Return Values:
- None

Omitted arguments are not overwritten. To delete a field supply an empty description.

There will also be filter-based modification interfaces, but these are TBD.

An experiment is deleted using:

Service: Experiments
Operation: removeExperiment
Input Parameters:
- Userid - the user making the request (the owner on success)
- Owner - the experiment owner
- Name - a string containing the experiment to delete
Return Values:
- None

After removeExperiment succeeds, experiment state is removed from the testbed completely.

Realizing and Releasing Experiments

To realize an experiment use:

Service: Experiments
Operation: realizeExperiment
Input Parameters:
- Userid - the user making the request (the owner on success)
- Owner - the experiment owner
- Name - a string containing the experiment to realize
- Notify - a flag, if true send the user a notification when realization is complete
Return Values:
- None

The user can either poll the experiment using viewExperiments and examining the StateDetail field or poll for a notification that the realization is complete.

If the realization fails - as indicated by the State variable in a viewExperiments response, the Log and FaultInfo fields characterize the error.

To release an experiment's resources:

Service: Experiments
Operation: releaseExperiment
Input Parameters:
- Userid - the user making the request (the owner on success)
- Owner - the experiment owner
- Name - a string containing the experiment to release
- Notify - a flag, if true send the user a notification when realization is complete
Return Values:
- None

After releaseExperiment succeeds, experiment state remains in the testbed, but containers are stopped and resources are returned to the testbed.