| 21 | | * Facilitate composition of functional and structural elements of the experiment based on stated and unstated constraints. The ELM workbenches allow creating and linking functional elements of the experiment without specifying the underlying structure and topology. Resolving the constraints to configure a set of realizable and executable experiment trials is a complex constraint satisfaction problem. |
| | 25 | * Inform design and analysis tools to obtain maximum information with the minimum number of experiment trials for a particular study. Every measured value in an experiment is fundamentally a |
| | 26 | random variable. Hence there are slight variations in the measurements during a trial even when all experimentation factors are kept constant. Hence to be able to characterize such stochastic behavior, |
| | 27 | it is necessary to execute multiple repetitions and identify confidence levels. Leveraging the tools in the analysis phase, feedback from the analysis phase can be used to control the number of |
| | 28 | required repetitions for statistically significant results. |
| 23 | | * Facilitate experiment monitoring and analysis for accuracy of results and availability of resources and services. ELM+SEER will enable monitoring the experiment configuration and performance of resources to ensure the experiment is executed correctly. While resource misconfiguration and failures are easier to spot, identifying "incorrect performance" of a resource or service is extremely hard. For stochastic processes as seen typically in networked systems, it is very important to be able to identify such experimentation errors as they can significantly impact results and bias measurements. |
| | 30 | * Facilitate composition of functional and structural elements of the experiment based on stated and unstated constraints. The ELM workbenches allow creating and linking functional elements |
| | 31 | of the experiment without specifying the underlying structure and topology. Resolving the constraints to configure a set of realizable and executable |
| | 32 | experiment trials is a complex constraint satisfaction problem. |
| | 33 | |
| | 34 | * Facilitate experiment monitoring and analysis for accuracy of results and availability of resources and services. ELM+SEER will enable monitoring the experiment configuration and performance of |
| | 35 | resources to ensure the experiment is executed correctly. While resource misconfiguration and failures are easier to spot, identifying "incorrect performance" of a resource or service is extremely |
| | 36 | hard. For stochastic processes as seen typically in networked systems, it is very important to be able to identify such experimentation errors as they can significantly impact results and bias |
| | 37 | measurements. |
| 25 | | * Enable reuse of experiment assets and artifacts. Reuse is driven by the ability to discover the workflows, scenarios, and data. The ELM environment will provide registry and registry views, along with (RDF-based, DAML+OIL) metadata to facilitate the discovery process. ELM will provide tools to index and search semantically rich descriptions and identify experimentation components including models, workflows, services and specialized applications. To promote sharing, ELM will provide annotation workbenches that allow experimenters to add sufficient metadata and dynamically link artifacts based on these annotations. |
| | 39 | * Enable reuse of experiment assets and artifacts. Reuse is driven by the ability to discover the workflows, scenarios, and data. The ELM environment will provide registry and registry views, along |
| | 40 | with (RDF-based, DAML+OIL) metadata to facilitate the discovery process. ELM will provide tools to index and search semantically rich descriptions and identify experimentation components |
| | 41 | including models, workflows, services and specialized applications. To promote sharing, ELM will provide annotation workbenches that allow experimenters to add sufficient metadata and dynamically |
| | 42 | link artifacts based on these annotations. |
| 27 | | * Support for multi-party experiments where a particular scenario can be personalized for a team in a ''appropriate'' way by providing restricted views and control over only certain aspects of the experiment. The registry view will allow the team to access only a restricted set of services. The analysis perspectives and views will present relevant animations and graphs to the team. Thus by personalizing a scenario view, the same underlying scenario, can be manipulated and observed in different ways by multiple teams. |
| | 44 | * Support for multi-party experiments where a particular scenario can be personalized for a team in a ''appropriate'' way by providing restricted views and control over only certain aspects of the |
| | 45 | experiment. The registry view will allow the team to access only a restricted set of services. The analysis perspectives and views will present relevant animations and graphs to the team. Thus by |
| | 46 | personalizing a scenario view, the same underlying scenario, can be manipulated and observed in different ways by multiple teams. |
| 37 | | |
| 38 | | |
| 39 | | |
| 40 | | |
| 41 | | |
| 42 | | |
| | 56 | '''Composition Phase ''' |
| | 57 | * defining the functional components and functional topology of the study. |
| | 58 | * defining the abstractions, models, parameters, and constraints for each functional component |
| | 59 | * identifying/defining the experiment workflow and invariants |
| | 60 | * identifying/defining the structural physical topology |
| | 61 | * Composing the experiment trials by resolving the constraints and exploring the parameter space |
| 48 | | |
| 49 | | |
| 50 | | |
| 51 | | |
| 52 | | |
| | 67 | '''Analysis Phase''' |
| | 68 | * analyzing completed trials (some trial may still be executing) |
| | 69 | * presenting results to experimenter |
| | 70 | * feedback parameters into the composition tools |
| | 71 | * annotate data and artifacts and store in the repositories |
