Optional: Coding 6

This optional section explores the ensemble class in greater depth. This section is intended for users who are interested in the full capabilities of the ensemble class, particularly users who are already familiar with using the DASH toolbox. You can skip this section if this is your first time using DASH. Some of these commands do appear later in the tutorial, but they will be discussed as they arise.

Goal

Practice using ensemble commands.

Step 1: Create ensemble object

You can use the ensemble command to create an ensemble object for a saved state vector ensemble. The syntax is:

obj = ensemble(filename)

where filename is the name of the saved ensemble file. The name should either be the name of a file on the active path, or an absolute file name. If you do not include a .ens extension, the method will apply one automatically.

If the stateVector object used to build the ensemble had a label, then the ensemble object will automatically be given that label. You can optionally use the second input to apply a different label to the object:

obj = ensemble(filename, label)

Tip

You can also use the ensemble.label command at any later point to re-label the ensemble.

Recall that we saved the demo state vector ensemble in a file named ntrend.ens. We’ll create an ensemble object for the saved ensemble:

file = "ntrend.ens";
ens = ensemble(file);

Inspecting the object:

disp(ens)

static ensemble with properties:

      Label: NTREND Temperature Demo
  Variables: T, T_index, T_monthly
     Length: 56161
    Members: 1156

we can see the saved state vector ensemble.

Step 2: Load data

Use the load command to load data from the saved ensemble. The syntax is:

[X, metadata] = obj.load
X

The first output is a numeric matrix. This is the loaded state vector ensemble. The columns of the matrix are ensemble members, and each row is a state vector element.

metadata

The second output is an ensembleMetadata object that describes the loaded array.

We’ll use the load command to load the saved state vector ensemble:

ens = ensemble("ntrend.ens");
[X, metadata] = ens.load;

Examining the data array:

size(X)

ans =
       56161        1156

we can see that the first output is a matrix with 56161 state vector elements, and 1156 ensemble members. The second output:

disp(metadata)

ensembleMetadata with properties:

      Label: NTREND Temperature Demo
  Variables: T, T_index, T_monthly
     Length: 56161
    Members: 1156

  Vector:
              T -  4320 rows   |   lon (144) x lat (30)
        T_index -     1 rows   |   lon (1) x lat (1)
      T_monthly - 51840 rows   |   lon (144) x lat (30) x time sequence (12)

records information about the loaded ensemble.

Step 3: Return Metadata

It’s often useful to return the metadata object for an ensemble without actually loading the ensemble. You can use the metadata command to return the metadata object. The syntax is:

metadata = obj.metadata

Here, we’ll return the metadata object for the saved ensemble:

% Load the metadata
ens = ensemble("ntrend");
metadata = ens.metadata;

% Display in console
disp(metadata)

ensembleMetadata with properties:

      Label: NTREND Temperature Demo
  Variables: T, T_index, T_monthly
     Length: 56161
    Members: 1156

  Vector:
              T -  4320 rows   |   lon (144) x lat (30)
        T_index -     1 rows   |   lon (1) x lat (1)
      T_monthly - 51840 rows   |   lon (144) x lat (30) x time sequence (12)

Step 4: Select Variables

Use the useVariables command to limit the ensemble object to a specific set of variables. Here the syntax is:

obj = obj.useVariables(variables)
variables

The input is a list of variables in the ensemble. You can either list variable names, or the indices of variables within the ensemble. Using -1 will reselect all the variables in the ensemble.

obj

The output is the updated ensemble object.

After using the useVariables command, the load command will only load data for the specified variables. Likewise, the metadata command will only return metadata for these variables.

We’ll use the useVariables command to limit the ensemble to the reconstruction targets - the T and T_index variables:

variables = ["T", "T_index"];
ens = ens.useVariables(variables);

Inspecting the object:

disp(ens)

static ensemble with properties:

      Label: NTREND Temperature Demo
  Variables: T, T_index
     Length: 4321
    Members: 1000

we can see that it only represents the two listed variables.

If we now call the load command and examine the output:

[X, metadata] = ens.load;
siz = size(X)

siz =
        4321        1156

we can see that the loaded matrix only includes the 4321 rows associated with the T and T_index variables. It does not include the remaining rows associated with the T_monthly variable.

Likewise the metadata object only includes information on the loaded variables:

disp(metadata)

ensembleMetadata with properties:

      Label: NTREND Temperature Demo
  Variables: T, T_index
     Length: 4321
    Members: 1156

  Vector:
            T - 4320 rows   |   lon (144) x lat (30)
      T_index -    1 rows   |   lon (1) x lat (1)

Step 5: Select Members

Use the useMembers command to limit the ensemble object to a specific set of ensemble members. Here the syntax is:

obj = obj.useMembers(members)
members

The input is a vector of indices pointing to specific ensemble members in the saved state vector ensemble. Both linear and logical indices are acceptable. Using -1 will reselect all saved ensemble members.

obj

The output is the updated ensemble object.

After using the useMembers command, the load command will only load data for the specified members. Likewise, the metadata command will only return metadata for these members.

This command is often combined with the ensembleMetadata.members command, which returns metadata for the members of an ensemble. This metadata can be used identify and select specific members within the ensemble. The base syntax for this command is:

metadata = obj.members(dimension)
dimension

The first input is the name of an ensemble dimension for which to return metadata.

metadata

The output is the metadata at the reference point for each ensemble member. The metadata will be a matrix with one row per ensemble member.

Here, we’ll limit the ensemble to 100 randomly selected ensemble members. We’ll use Matlab’s randsample command to select 100 members from the 1156 member ensemble:

% (Reset the random number generator to make the demo reproducible)
rng('default')

% Select 100 members at random
members = randsample(1156, 100);

% Create an ensemble object that uses the 100 members
ens = ensemble('ntrend');
ens = ens.useMembers(members);

Examining the ensemble object:

disp(ens)

static ensemble with properties:

      Label: NTREND Temperature Demo
  Variables: T, T_index, T_monthly
     Length: 56161
    Members: 100

we see it now represents an ensemble with 100 members.

If we now call the load command and examine the output:

[X, metadata] = ens.load;
siz = size(X)

siz =
       56161         100

we can see that the loaded matrix only include the 100 columns (ensemble members) associated with the object. Likewise the metadata object only records values for 100 members:

disp(metadata)

  ensembleMetadata with properties:

        Label: NTREND Temperature Demo
    Variables: T, T_index, T_monthly
       Length: 56161
      Members: 100

We can also use the ensembleMetadata.members command to see which members were selected:

% Get the time metadata for the 100 members
time = metadata.members("time");

% Display the size and metadata
siz = size(time)
disp(time)

siz =
   100     1

time =
    100×1 datetime array

     15-Jan-1014
     15-Jan-1634
     15-Jan-1635
     ...
     15-Jan-1562
     15-Jan-1010
     15-Jan-0962

We can see that the metadata includes metadata for 100 ensemble members, and that the ensemble members are randomly selected from the 1156 January reference points.

Here, we’ll limit the ensemble to members from the pre-industrial era - that is, ensemble members from before 1850. We’ll use the ensembleMetadata.members method to help locate these members:

% Build an ensemble object and get its metadata object
ens = ensemble('ntrend');
metadata = ens.metadata;

% Get the time metadata for each ensemble member and locate preindustrial members
time = metadata.members("time")
preindustrial = year(time) < 1850;

% Only use the preindustrial ensemble members
ens = ens.useMembers(preindustrial);

Examining the ensemble object:

disp(ens)

static ensemble with properties:

      Label: NTREND Temperature Demo
  Variables: T, T_index, T_monthly
     Length: 56161
    Members: 1000

we can see it represents an ensemble with 1000 members. We can use the updated object’s ensembleMetadata to verify that the ensemble uses the 1000 preindustrial members:

metadata = ens.metadata;
time = metadata.members("time")

1000×1 datetime array

 15-Jan-0850
 15-Jan-0851
 15-Jan-0852
 ...
 15-Jan-1847
 15-Jan-1848
 15-Jan-1849

Step 6: Evolving Ensemble

You can use the evolving command to implement an evolving ensemble. Each ensemble in an evolving set is built from a different selection of ensemble members. The syntax for the command is:

obj = obj.evolving(members)
members

The first input indicates which ensemble members to use in each ensemble of an evolving set. This input is a matrix of indices. Each column lists the members for a particular ensemble in the evolving set. Both linear and logical indices are accepted, but each ensemble should have the same number of members.

obj

The output is the updated ensemble object.

You can also use the optional second input to provide a set of labels for the ensembles in the evolving set:

obj = obj.evolving(members, labels)

The labels input should be a vector of strings with one label per ensemble.

Tip

You can also use the evolvingLabels command to apply labels to the evolving ensembles.

This command is often combined with the ensembleMetadata.members command, which helps locate members for specific ensembles. See the section above for its syntax.

After using the evolving command, the load command will return a 3D data array, rather than a data matrix. The rows and columns are the same as before, and elements along the third dimension correspond to ensembles in the evolving set. The output metadata will become a vector of metadata objects with one object per ensemble in the set. The metadata command will similarly return a vector of metadata objects.

Furthermore, you can now provide an optional input to the load and metadata commands. The syntax becomes:

[X, metadata] = obj.load(ensembles)
metadata = obj.metadata(ensembles)

and allows you to return values for specific ensembles in the evolving set. The ensembles input is a list of ensembles for which to return values. You can either list the labels associated with particular ensembles, or the indices of ensembles in the evolving set.

We’ll design an evolving ensemble with three individual ensembles. Each ensemble in the evolving set will be built from a different set of 100 ensemble members. Specifically, the three ensembles will correspond to the years 1200-1299, 1800-1899, and 1900-1999. We’ll label the individual ensembles as “Preindustrial”, “Mixed”, and “Modern”

% Build an ensemble object and get its metadata
ens = ensemble('ntrend');
metadata = ens.metadata;

% Get the time metadata for the ensemble members.
time = metadata.members('time');
time = year(time);

% Select members for the three ensembles
pi = ismember(time, 1200:1299);
mixed = ismember(time, 1800:1899);
modern = ismember(time, 1900:1999);

% Design the evolving ensemble
members = [pi, mixed, modern];
labels = ["Preindustrial" ,"Mixed", "Modern"];
ens = ens.evolving(members, labels);

Examining the ensemble object:

disp(ens)

evolving ensemble with properties:

      Label: NTREND Temperature Demo
  Variables: T, T_index, T_monthly
     Length: 56161
    Members: 100  (per ensemble)

  Evolving Ensembles: 3
      1. Preindustrial
      2. Mixed
      3. Modern

we can see that the object now represents an evolving ensemble with 3 individual ensembles in the evolving set.

If we call the load command and examine the output:

[X, metadata] = ens.load;
siz = size(X)

siz =
   56161         100           3

we can see that the loaded data array has 3 elements along the third dimension - one element per ensemble in the evolving set. Additionally, the array has 100 columns, so each individual ensemble is built from 100 ensemble members.

Examining the metadata:

disp(metadata)

3x1 ensembleMetadata array

  Labels:

  "Preindustrial"
  "Mixed"
  "Modern"

we can see that it includes 3 metadata objects - one per ensemble. Finally, we can use the loaded metadata objects to verify the members in each ensemble. For example, for the first (preindustrial) ensemble:

time = metadata(1).members

time =
    100×1 datetime array

     15-Jan-1200
     15-Jan-1201
     15-Jan-1202
     ...
     15-Jan-1297
     15-Jan-1298
     15-Jan-1299

We can also use the load method to load specific ensembles within the evolving set. Here, we’ll load the Preindustrial and Modern, but not the Mixed (1800-1899) ensemble:

[X, metadata] = ens.load(["Preindustrial", "Modern"])
siz = size(X)

siz =
   56161         100           2

disp(metadata)

2x1 ensembleMetadata array

  Labels:

  "Preindustrial"
  "Modern"

Full Demo

% Create ensemble object
file = 'ntrend.ens';
ens = ensemble(file);

% Select variables
variables = ["T", "T_index"];
ens = ens.useVariables(variables);

% Select ensemble members
members = randsample(1000, 100);
ens = ens.useMembers(members);

% Load
[X, metadata] = ens.load;

%%%%%%%%

% Get metadata for ensemble members
ens = ensemble('ntrend');
metadata = ens.metadata;
time = year(metadata.members('time'));

% Locate ensemble members
pi = ismember(time, 1200:1299);
mixed = ismember(time, 1800:1899);
modern = ismember(time, 1900:1999);

% Design the evolving ensemble
members = [pi, mixed, modern];
labels = ["Preindustrial" ,"Mixed", "Modern"];
ens = ens.evolving(members, labels);

% Load specific ensembles in evolving set
[X, metadata] = ens.load(["Preindustrial","Modern"]);