Trade Space
Trade space analysis of constellations.
A trade study declares a design space (DesignSpace), evaluates its designs in replicates that draw its uncertain parameters (Study, or ObservationStudy for the observations, deployment, costs, and failures of constellations) into results (a row per replicate), judges designs by criteria (Criteria) on summaries of the results (summarize, a row per design) and their Pareto efficiency (pareto_front), and settles the uncertainty of its replicates (refine, confirm), or finds efficient designs of spaces too large to enumerate (screen, optimize_optuna, optimize_pymoo).
This package exports the functions and classes of trade studies, and the outcomes of their steps (such as Refinement), for type hints. The parts from which they are made (such as the layouts of constellations, the simulation of missions, and fingerprints) are in its modules.
@author: Paul T. Grogan <paul.grogan@asu.edu>
The Trade Space Tutorial shows each step on a study of the observations of constellations, one step at a time.
Steps of a Trade Study
A trade study takes a few steps, each of which takes the outcome of the one before:
from tatc.tradespace import (
Criteria, DesignSpace, Integer, ResultStore, Study, Uncertain,
confirm, refine, summarize,
)
space = DesignSpace(planes=Integer(1, 4), per_plane=Integer(1, 4), seed=Uncertain.seed())
study = Study(space, build, evaluate, replicates=5, store=ResultStore("results.sqlite"))
results = study.run() # a row per design and replicate
criteria = Criteria({"satellites": "min", "coverage": "max"})
summary = summarize(results, criteria) # a row per design, with `pareto`
front = summary[summary.pareto]
refinement = refine(study, criteria, results) # replicates where efficiency is uncertain
confirmation = confirm(study, criteria, refinement.results) # the front, with new replicates
Define a design space (Design Spaces): design variables, choices and repetitions, constraints, and uncertain parameters, which each replicate of a design draws.
Evaluate designs with a study (Studies): a function builds each realized design and another evaluates it, in parallel, with a store of the evaluations. An ObservationStudy evaluates the observations of constellations, and their deployment, costs, and failures (Constellations and Missions).
Judge designs by criteria (Criteria and Summaries): objectives and requirements on statistics of their replicates, summarized with a row per design, and their Pareto efficiency.
Settle the uncertainty of the replicates, or search spaces too large to enumerate (Refinement and Searches). Each of these steps takes a study and criteria, and returns its results, the summary of its designs, the designs themselves, for the next step, and its criteria (as an outcome, such as a Refinement, see Refinement and Searches).
Results and Summaries
A study gives results: a row per evaluation (a design in a replicate, and a scenario), with the key of the design (design), the replicate, the values of the design by path (see Design.flat), whether it is feasible, and its metrics. A summary has a row per design (and per group of the criteria’s by): statistics over its feasible replicates, the number of them (replicates), the share of its replicates that are feasible (which requirements may name), and the values of the design, and, of criteria, the violation of their requirements, whether the design meets them (meets_requirements), and whether it is efficient (pareto). Functions take one or the other:
Take results (a row per replicate) |
Take summaries (a row per design) |
|---|---|
summarize, pareto_probability, hypervolume_progress, robustness, exceedance, refine, confirm, plot_means |
pareto_front, pareto_ranks, hypervolume, sensitivity, main_effects, plot_designs |
The functions of summaries raise an error if a design has several rows, as in the results of a study: summarize them first. The steps that evaluate designs (such as confirm and attribute_variance) take designs, their keys, or rows of either (see DesignSpace.designs_of).
Columns
Columns named after a metric or a statistic take a suffix after a space, and columns that TAT-C defines are named in snake case:
statistics over the replicates of a design: named by the criteria, or by convention if given without names (see Statistic.name, such as “cost p70” for Quantile(“cost”, 0.7)), the means of metrics by their names, each mean followed by its spread, <metric> std and <metric> se (by summarize, and by pareto_probability, which gives the standard error of an objective that is not a mean by its bootstrap, left out where they are zero for every design), and of confirmed designs, <statistic> selected (by the replicates that selected them) and <objective> optimism. A statistic other than the mean of a metric must not have the name of a column of the results.
statistics within a replicate, over the mission of a design (see ObservationStudy): <metric> mission (the mean over the phases of the mission), <metric> time (when the phases first meet a requirement), and, over the histories of its failures, <column> histories p<percent>, <column> histories std, and P(<requirement>) histories.
columns of judgments: violation, meets_requirements, pareto, pareto_rank, pareto_probability, requirements_probability, pareto_trace, max_regret, and the <scenario> regret of each scenario.
Designs and Their Variants
Results keep the values of designs, from which DesignSpace.designs_of rebuilds them, so that the steps of a study take designs as Design objects, their keys, or rows of a summary or of results. A design gives its values by attribute (d.layout.planes) or by the path of a column of the results (d[“layout.walker.planes”], see Values), and a study realizes it as it evaluates it (Study.realize, and, of an ObservationStudy, satellites_of, observations_of, and mission_of, whose Mission gives its deployment, phases, cost estimate, and histories):
confirmation = confirm(study, criteria, results, summary[summary.pareto])
A variant of a design space adds or replaces children (DesignSpace.extend), and its designs take the decisions of a design of the other (DesignSpace.design). A variant of a study replaces some of its arguments (Study.replace):
policies = space.extend(policy=policy_space(orbit_spares=Integer(0, 1)))
designs = [policies.design(d, policy=p) for d in front_designs for p in policy_designs]
results = study.replace(space=policies, replicates=20).run(designs)
The parts of an ObservationStudy (such as its targets and mission plan)
refer to values of designs by a Reference (see
tatc.tradespace.references.Reference): a name or path of a
value, or a function of the realized design.
Stores of Results
A ResultStore keeps the evaluations of studies, keyed by the fingerprint of each study’s definition (see Study.fingerprint), so that a study that is run again takes its evaluations from the store, and variants of a study (other settings, design spaces, or scenarios) share a store without mixing their evaluations. A namespace of the store identifies only what fingerprints cannot see (such as data that the functions read from files). A study warns if the store has evaluations of the same study with other code (of its functions, of TAT-C, or of Python), which it does not reuse (Study.prune removes them, and not the evaluations of other studies that share the store).