mlpy.mdp.stateaction.MDPPrimitive

class mlpy.mdp.stateaction.MDPPrimitive(features, name=None)[source]

Bases: object

A Markov decision process primitive.

The base class for State and Action. Primitives are represented by a list of features. They optionally can have a name.

Parameters:

features : array_like, shape (nfeatures,)

List of features, where nfeatures is the number of features identifying the primitive.

name : str, optional

The name of the primitive. Default is “”.

Raises:

ValueError

If the feature array is not one-dimensional.

Notes

Use the description to encode action information. The information should contain the list of all available feature combinations, the name of each feature.

Examples:

A description of an action with three possible discrete actions:

{
    "out": {"value": [-0.004]},
    "in": {"value": [0.004]},
    "kick": {"value": [-1.0]}
}

A description of an action with one possible continuous action with name move, a value of * allows to find the action for every feature array. Additional information encodes the feature name together with its index into the feature array are given for each higher level element of feature array:

{
    "move": {
        "value": "*",
        "descr": {
            "LArm": {"dx": 0, "dy": 1, "dz": 2},
            "RArm": {"dx": 3, "dy": 4, "dz": 5},
            "LLeg": {"dx": 6, "dy": 7, "dz": 8},
            "RLeg": {"dx": 9, "dy": 10, "dz": 11},
            "Torso": {"dx": 12, "dy": 13, "dz": 14}
        }
    }
}

Similarly, a continuous state can be encoded as follows, which identifies the name of each feature together with its index into the feature array:

{
    "LArm": {"x": 0, "y": 1, "z": 2},
    "RArm": {"x": 3, "y": 4, "z": 5},
    "LLeg": {"x": 6, "y": 7, "z": 8},
    "RLeg": {"x": 9, "y": 10, "z": 11},
    "Torso": {"x": 12, "y": 13, "z": 14}
}

A discrete state can be encoded by identifying the position of each feature:

{
    "image x-position": 0,
    "displacement (mm)": 1
}

Alternatively, the feature can be identified by a list of features, giving he positional description:

["image x-position", "displacement (mm)"]

Rather then setting the attributes directly, use the methods set_nfeatures, set_dtype, set_description, set_discretized, set_minmax_features, and set_states_per_dim in order to enforce type checking.

Attributes

name The name of the MDP primitive.
dtype
nfeatures (int) The number of features.
discretized (bool) Flag indicating whether the features are discretized or not.
min_features (list) The minimum value for each feature.
max_features (list) The minimum value for each feature.
states_per_dim (list) The number of states per dimension.
description (dict) A description of the features.

Methods

DTYPE_FLOAT
DTYPE_INT
DTYPE_OBJECT
decode(_repr) Decodes the state into its original representation.
discretize() Discretizes the state.
dtype
encode() Encodes the state into a human readable representation.
get() Return the feature array.
key_to_index(key) Maps internal name to group index.
next()
set(features) Sets the feature array to the given array.
set_description(descr) Set the feature description.
set_discretized([val]) Sets the discretized flag.
set_dtype([value]) Set the feature’s data type.
set_minmax_features(_min, _max) Sets the minimum and maximum value for each feature.
set_nfeatures(n) Set the number of features.
set_states_per_dim(nstates) Sets the number of states per feature.
tolist() Returns the feature array as a list.