Simulation

The simulation API separates four jobs: construct a model, advance it, collect selected values, and compare those values with observations.

FunctionMain roleResult
instanceconstruct and initialize one modela mutable system instance
simulateconstruct fresh instances and run themone DataFrame, or one per layout
simulate!continue an existing instanceone DataFrame, or one per layout
evaluatejoin observations and estimates and calculate metricsone number or a tuple
calibratesearch parameter bounds against those metricsa Config or Pareto frontier

For a first simulation, read Model Execution and Run Simulations and Shape Output. This page records the complete public call patterns and the less common options exercised by the test suite.

Construct one model with instance

instance(SystemType;
    config = (),
    options = (),
    seed = nothing,
)

instance normalizes config, calls the generated system constructor, and runs the initialization update. It returns the model itself, not a table.

  • config supplies values declared as parameters.
  • options is a named tuple of constructor resources, typically for extern or override declarations that are not ordinary parameters.
  • seed initializes Julia's random generator before configuration values are sampled and before the system is constructed.

Prefer a system type as a configuration key when it is in scope. Type keys check parameter names and convert compatible units immediately. Symbol or string keys remain useful when loading configuration before the system package, but validation is then deferred until construction.

Advance one update with update!

update!(instance)

One call performs one generated Cropbox update cycle in dependency order. It is useful in tests and tightly controlled interactive code:

s = instance(Model; config)
update!(s)
value(s.context.clock.time)

For normal runs, prefer simulate or simulate!; they also handle stopping, snapshots, callbacks, output selection, and progress. Do not manually update an instance while a simulation callback is traversing the same instance.

update! is also an internal extension point with stage arguments, and Cropbox uses the same exported name for updating a plot's option store. Those methods are implementation-facing; model code should normally call the one-argument system form or the documented visualize! function.

Run fresh models with simulate

simulate(SystemType; keywords...) -> DataFrame
simulate(SystemType, layout; keywords...) -> Vector{DataFrame}
simulate(SystemType, layout, configs; keywords...) -> Vector{DataFrame}
simulate(; system = SystemType, keywords...) -> DataFrame
simulate(f, SystemType, ...; keywords...) -> DataFrame

The last positional callback form is do-block syntax for snatch:

simulate(Model; stop = 10u"d") do rows, model
    rows[1][:count] = length(model.organs')
end

With no stop, snap, snatch, or callback, simulate returns the initialized snapshot and performs no time-step update. This fast snapshot form is useful for static response models and parameter grids.

Construction keywords

KeywordDefaultMeaning
config()one configuration, or a shared base for configs
configs[]scenario patches or complete configurations
parameters()a compact parameter sweep with automatic metadata
options()named constructor resources passed to every instance
seednothingrandom seed reset before each fresh instance

When both config and configs are present, Cropbox merges the shared config into every entry of configs, with the scenario entry taking precedence. parameters is a convenience sweep: Cropbox expands its iterable values and adds the selected parameter values as metadata. Do not combine nonempty parameters and configs; construct the configurations explicitly when a design needs both.

simulate(ResponseModel;
    config = Clock => :step => 1u"d",
    parameters = ResponseModel => :temperature => 5:5:30,
    target = :rate,
)

Batch runs may use Julia threads. Keep callbacks free of unsynchronized writes to shared state. A seed makes each scenario reproducible, but it is reset to the same value for every scenario; run an explicit seed loop when replicates must use distinct random streams.

Output layout

A simple call supplies one layout through keywords:

simulate(Model;
    base = nothing,
    index = :time => "context.clock.time",
    target = [:LAI, :biomass],
    meta = [:Treatment, :replicate => 1],
    stop = 100u"d",
)
KeywordDefaultMeaning
basenothingsystem or bundle relative to which selectors are resolved
indexclock timecolumns that identify output rows
targetsimple root valuesmodel values to collect
metanoneconfiguration values or constants copied to every row

Selectors accept:

  • a symbol such as :LAI;
  • a string path such as "calendar.date";
  • a renamed pair such as :date => "calendar.date";
  • a tuple or vector of selectors;
  • "*" for root fields or "soil.*" for one nested system's fields.

The left side of a pair is the output column and the right side is the model path. Explicit targets make downstream code stable when a model later gains new variables. Wildcards are best reserved for inspection.

Only simple scalar output is collected automatically: numbers, symbols, strings, and date/time values. Arrays, dictionaries, tuples, DataFrames, and nested systems are omitted. Add a scalar summary declaration or use snatch for those objects.

meta=:Treatment copies every configured entry under that system name. meta=(:Treatment, :replicate => 1) combines configured values with an explicit constant column. Metadata comes from configuration, not from the current state of a changing model variable.

Several layouts in one run

Pass a vector of named tuples when different tables should be collected from the same update sequence:

layout = [
    (index = :time, target = [:LAI, :biomass]),
    (base = :soil, index = :depth, target = [:water, :root_length]),
    (target = :mature, meta = (:site => "A",)),
]

tables = simulate(Model, layout; config, stop = :finished)

Each layout may contain base, index, target, and meta; omitted entries use the same defaults as a simple call. The result order matches the layout order. With a separate positional configs vector, each returned table combines the corresponding layout across all scenarios.

Stop and snapshot conditions

stop determines how long the model advances. It accepts:

FormMeaning
integer or real numberthat many update calls
time quantityenough updates to reach or pass the duration, using Clock.step
symbol or stringread a numeric, time, or Boolean model variable
function model -> valuecalculate the same kinds of condition
Boolean model conditionkeep updating until it becomes true

A numeric stop is interpreted once as an update count. A Boolean stop is tested before every update. If the stop threshold is already true after initialization, no update is made.

snap decides which initialized or updated states become output rows:

FormMeaning
nothingsave every state
time quantitysave at elapsed-time multiples measured from Clock.init
symbol or stringsave when that model value is true
function model -> Boolsave when the callback is true

Clock.step still controls every numerical update. snap=1u"d" does not turn an hourly model into a daily model; it only keeps every 24th hourly state. The initialized state is included whenever the snapshot condition is true, so a run with stop=10 normally has 11 rows.

Custom snapshot processing

snatch(rows, model) runs for every saved state. rows contains one extracted row per matching base-system item and may be edited, extended, or emptied. This is the intended extension point for dynamic structures such as root architectures.

callback(model, simulation_layout) runs after a saved state has been processed. The second argument is Cropbox's internal simulation accumulator. It is lower level and more sensitive to implementation changes than snatch; prefer snatch unless coordination with the accumulator is necessary. The current callback path runs after updated snapshots, not after the initial snapshot; snatch receives both initial and updated snapshots.

Supplying either callback disables the fastest column-oriented collection path. That is normally insignificant for complex callback work, but it matters for large scalar-only sweeps.

Output formatting

KeywordDefaultMeaning
nounitfalsestrip units from returned columns
longfalsestack target columns into variable/value rows
verbosetrueshow a progress display

nounit=true changes only the returned table. Model calculations remain unit-aware. Long format preserves index columns, stacks all remaining columns, and sorts by the index. Request it only when a plotting or statistics workflow needs tidy long data, since it can greatly increase the row count.

Continue one model with simulate!

simulate!(instance; layout_keywords..., run_keywords...) -> DataFrame
simulate!(instance, layout; run_keywords...) -> Vector{DataFrame}
simulate!(f, instance, ...; run_keywords...) -> DataFrame

simulate! continues the exact instance passed to it. It accepts output, stop/snapshot, callback, and formatting keywords, but not a new config, options, or seed. Use it for deliberate staged runs or manual management; use simulate(SystemType, ...) for independent treatments and replicates.

s = instance(Model; config, seed = 1)
first_stage = simulate!(s; stop = :emerged, target = :biomass)
second_stage = simulate!(s; stop = :mature, target = :biomass)

The second stop is evaluated from the already advanced state. The two returned tables are separate. Each call saves its current state first when the snapshot condition is true, so the stage boundary can appear in both tables.

Compare data with evaluate

evaluate(observations, estimates;
    index,
    target,
    metric = :rmse,
)

evaluate(SystemType, observations;
    config = (), configs = [],
    index = nothing,
    target,
    metric = :rmse,
    simulation_keywords...,
)

For two tables, target=:observed => :estimated maps different column names. For a system, the same name is assumed unless a pair is supplied. Multiple targets return a tuple in target order.

Cropbox normalizes compatible index units, inner-joins rows on the index, drops missing target pairs in the two-table form, and then applies the metric. It does not compare rows merely because they have the same position.

SymbolMetricResult scale
:rmseroot mean square errortarget unit
:nrmseRMSE divided by observation meandimensionless
:rmsperoot mean square percentage errordimensionless
:maemean absolute errortarget unit
:mapemean absolute percentage errordimensionless
:efNash–Sutcliffe efficiencydimensionless
:drrefined index of agreementdimensionless

metric may also be a function (estimate, observation) -> score. Relative metrics need special care near zero. Efficiency metrics need enough variation in the observations to define their denominator.

When evaluating several configurations, residuals are combined across their matching rows before one score per target is calculated. Give environment or treatment columns in index when otherwise identical dates must remain distinct.

Search parameters with calibrate

calibrate(SystemType, observations;
    config = (), configs = [],
    index = nothing,
    target,
    parameters,
    metric = :rmse,
    weight = nothing,
    pareto = false,
    optim = (),
    simulation_keywords...,
)

parameters has the shape of a configuration, but each value is a two-value search bound:

parameters = Model => (
    base_temperature = (-5, 15),
    thermal_requirement = (100, 2000),
)

Plain bounds inherit the unit declared by each parameter and are converted before optimization. This is usually the clearest form. When explicit units are needed, put them on the values (for example (-5u"°C", 15u"°C")) or multiply a vector such as [0, 2]u"yr^-1"; (0, 2)u"yr^-1" is not valid Julia syntax. The result is a Config containing the selected values in model units.

Cropbox uses BlackBoxOptim.jl differential-evolution methods. optim is a named tuple forwarded to bboptimize; Cropbox defaults are MaxSteps=5000 and TraceInterval=10. Immediately before calling bboptimize, Cropbox calls Random.seed!(0), so optimizer initialization is deterministic. This is framework behavior rather than a BlackBoxOptim option. Set an explicit search budget in scripts so runtime does not depend on defaults.

With one target, calibration minimizes that target's metric. With several targets, it uses a multi-objective method. weight changes how a single compromise is chosen from objective values. pareto=true returns an ordered mapping from objective tuples to configurations along the Pareto frontier instead of one Config.

Local or gradient-based methods are not selected through calibrate. When they are appropriate, construct and test an explicit objective with evaluate and pass it to an optimization package such as Optim.jl.

Do not pass nonempty config and configs together to evaluate or calibrate. For several environments, make each entry of configs complete and include environment-identifying columns in index. See Evaluate and Calibrate Models for validation splits, residual checks, and reporting practice.

API docstrings

Cropbox.instanceFunction
instance(S; <keyword arguments>) -> S

Make an instance of system S with an initial condition specified in configuration and additional options.

See also: @config, simulate

Arguments

  • S::Type{<:System}: type of system to be instantiated.

Keyword Arguments

  • config=(): configuration containing parameter values for the system.
  • options=(): keyword arguments passed down to the constructor of S; named tuple expected.
  • seed=nothing: random seed initialized before parsing configuration and making an instance.

Examples

julia> @system S(Controller) begin
           a => 1 ~ preserve(parameter)
           b(a) ~ accumulate
       end;

julia> instance(S)
S
  context = <Context>
  config = <Config>
  a = 1.0
  b = 0.0
source
Cropbox.simulateFunction
simulate([f,] S[, layout, [configs]]; <keyword arguments>) -> DataFrame

Run simulations by making instance of system S with given configuration to generate an output in the form of DataFrame. layout contains a list of variables to be saved in the output. A layout of single simulation can be specified in the layout arguments placed as keyword arguments. configs contains a list of configurations for each run of simulation. Total number of simulation runs equals to the size of configs. For a single configuration, config keyword argument may be preferred. Optional callback function f allows do-block syntax to specify snatch argument for finer control of output format.

See also: instance, @config

Arguments

  • S::Type{<:System}: type of system to be simulated.
  • layout::Vector: list of output layout definition in a named tuple (; base, index, target, meta).
  • configs::Vector: list of configurations for defining multiple runs of simluations.

Keyword Arguments

Layout

  • base=nothing: base system where index and target are populated; default falls back to the instance of S.
  • index=nothing: variables to construct index columns of the output; default falls back to context.clock.time.
  • target=nothing: variables to construct non-index columns of the output; default includes most variables in the root instance.
  • meta=nothing: name of systems in the configuration to be included in the output as metadata.

Configuration

  • config=(): a single configuration for the system, or a base for multiple configurations (when used with configs).
  • configs=[]: multiple configurations for the system.
  • seed=nothing: random seed for resetting each simulation run.

Progress

  • stop=nothing: condition checked before calling updates for the instance; default stops with no update.
  • snap=nothing: condition checked to decide if a snapshot of current update is saved in the output; default snaps all updates.
  • snatch=nothing: callback for modifying intermediate output; list of DataFrame D collected from current update and the instance of system s are provided.
  • verbose=true: shows a progress bar.

Format

  • nounit=false: remove units from the output.
  • long=false: convert output table from wide to long format.

Examples

julia> @system S(Controller) begin
           a => 1 ~ preserve(parameter)
           b(a) ~ accumulate
       end;

julia> simulate(S; stop=1)
2×3 DataFrame
 Row │ time       a        b
     │ Quantity…  Float64  Float64
─────┼─────────────────────────────
   1 │    0.0 hr      1.0      0.0
   2 │    1.0 hr      1.0      1.0
source
Cropbox.simulate!Function
simulate!([f,] s[, layout]; <keyword arguments>) -> DataFrame

Run simulations with an existing instance of system s. The instance is altered by internal updates for running simulations.

See also: simulate

source
Cropbox.evaluateFunction
evaluate(S, obs; <keyword arguments>) -> Number | Tuple

Compare output of simulation results for the given system S and observation data obs with a choice of evaluation metric.

Arguments

  • S::Type{<:System}: type of system to be evaluated.
  • obs::DataFrame: observation data to be used for evaluation.

Keyword Arguments

Configuration

  • config=(): a single configuration for the system (can't be used with configs).
  • configs=[]: multiple configurations for the system (can't be used with config).

Layout

  • index=nothing: variables to construct index columns of the output; default falls back to context.clock.time.
  • target: variables to construct non-index columns of the output.

Evaluation

  • metric=nothing: evaluation metric (:rmse, :nrmse, :mae, :mape, :ef, :dr); default is RMSE.

Remaining keyword arguments are passed down to simulate with regard to running system S.

See also: simulate, calibrate, @config

Examples

julia> @system S(Controller) begin
           a => 19 ~ preserve(u"m/hr", parameter)
           b(a) ~ accumulate(u"m")
       end;

julia> obs = DataFrame(time=10u"hr", b=200u"m");

julia> configs = @config !(:S => :a => [19, 21]);

julia> evaluate(S, obs; configs, target=:b, stop=10u"hr")
10.0 m
source
evaluate(obs, est; <keyword arguments>) -> Number | Tuple

Compare observation data obs and estimation data est with a choice of evaluation metric.

Arguments

  • obs::DataFrame: observation data to be used for evaluation.
  • est::DataFrame: estimated data from simulation.

Keyword Arguments

Layout

  • index: variables referring to index columns of the output.
  • target: variables referring to non-index columns of the output.

Evaluation

  • metric=nothing: evaluation metric (:rmse, :nrmse, :mae, :mape, :ef, :dr); default is RMSE.

See also: evaluate

Examples

julia> obs = DataFrame(time = [1, 2, 3]u"hr", b = [10, 20, 30]u"g");

julia> est = DataFrame(time = [1, 2, 3]u"hr", b = [10, 20, 30]u"g", c = [11, 19, 31]u"g");

julia> evaluate(obs, est; index = :time, target = :b)
0.0 g

julia> evaluate(obs, est; index = :time, target = :b => :c)
1.0 g
source
Cropbox.calibrateFunction
calibrate(S, obs; <keyword arguments>) -> Config | OrderedDict

Obtain a set of parameters for the given system S that simulates provided observation obs closely as possible. A multitude of simulations are conducted with a differing combination of parameter sets specified by the range of possible values and the optimum is selected based on a choice of evaluation metric. Internally, differential evolution algorithm from BlackboxOptim.jl is used.

Arguments

  • S::Type{<:System}: type of system to be calibrated.
  • obs::DataFrame: observation data to be used for calibration.

Keyword Arguments

Configuration

  • config=(): a single base configuration for the system (can't be used with configs).
  • configs=[]: multiple base configurations for the system (can't be used with config).

Layout

  • index=nothing: variables to construct index columns of the output; default falls back to context.clock.time.
  • target: variables to construct non-index columns of the output.

Calibration

  • parameters: parameters with a range of boundary values to be calibrated within.
  • metric=nothing: evaluation metric (:rmse, :nrmse, :mae, :mape, :ef, :dr); default is RMSE.

Multi-objective

  • weight=nothing: weights for calibrating multiple targets; default assumes equal weights.
  • pareto=false: returns a dictionary containing Pareto frontier instead of a single solution satisfying multiple targets.

Advanced

  • optim=(): extra options for BlackBoxOptim.bboptimize.

Remaining keyword arguments are passed down to simulate with regard to running system S.

See also: simulate, evaluate, @config

Examples

julia> @system S(Controller) begin
           a => 0 ~ preserve(parameter)
           b(a) ~ accumulate
       end;

julia> obs = DataFrame(time=10u"hr", b=200);

julia> p = calibrate(S, obs; target=:b, parameters=:S => :a => (0, 100), stop=10)
...
Config for 1 system:
  S
    a = 20.0
source