mlpy.mdp.stateaction.Action

class mlpy.mdp.stateaction.Action(features, name=None)[source]

Bases: mlpy.mdp.stateaction.MDPPrimitive

Representation of an action.

Actions are represented by an array of features.

Parameters:

features : array_like, shape (nfeatures,)

List of features, where nfeatures is the number of features identifying the primitive.

name : str, optional

The name of the primitive. Default is ‘’.

Notes

Use the description to encode action information. The information should contain the list of all available feature combinations, the name of each feature.

Examples:

A description of an action with three possible discrete actions:

{
    "out": {"value": [-0.004]},
    "in": {"value": [0.004]},
    "kick": {"value": [-1.0]}
}

A description of an action with one possible continuous action with name move, a value of * allows to find the action for every feature array. Additional information encodes the feature name together with its index into the feature array are given for each higher level element of feature array:

{
    "move": {
        "value": "*",
        "descr": {
            "LArm": {"dx": 0, "dy": 1, "dz": 2},
            "RArm": {"dx": 3, "dy": 4, "dz": 5},
            "LLeg": {"dx": 6, "dy": 7, "dz": 8},
            "RLeg": {"dx": 9, "dy": 10, "dz": 11},
            "Torso": {"dx": 12, "dy": 13, "dz": 14}
        }
    }
}

Similarly, a continuous state can be encoded as follows, which identifies the name of each feature together with its index into the feature array:

{
    "LArm": {"x": 0, "y": 1, "z": 2},
    "RArm": {"x": 3, "y": 4, "z": 5},
    "LLeg": {"x": 6, "y": 7, "z": 8},
    "RLeg": {"x": 9, "y": 10, "z": 11},
    "Torso": {"x": 12, "y": 13, "z": 14}
}

A discrete state can be encoded by identifying the position of each feature:

{
    "image x-position": 0,
    "displacement (mm)": 1
}

Alternatively, the feature can be identified by a list of features, giving he positional description:

["image x-position", "displacement (mm)"]

Rather then setting the attributes directly, use the methods set_nfeatures, set_dtype, set_description, set_discretized, set_minmax_features, and set_states_per_dim in order to enforce type checking.

Examples

>>> Action.set_description({'LArm': {'dx': 0, 'dy': 1, 'dz': 2}
...                         'RArm': {'dx': 3, 'dy': 4, 'dz': 5}})

This description identifies the features to be the delta x-y-z-position of the left and the right arm. The position into the feature array is given by the integer numbers.

>>> def my_key_to_index(key)
...     return {
...         "dx": 0,
...         "dy": 1,
...         "dz": 2
...     }[key]
...
>>> Action.key_to_index = staticmethod(my_key_to_index)

This defines a mapping for each key.

>>> action = [0.1, 0.4, 0.3. 4.6. 2.5. 0.9]
>>>
>>> mapping = Action.description['LArm']
>>>
>>> larm = np.zeros[len(mapping.keys())]
>>> for key, axis in mapping.iteritems():
...     larm[Action.key_to_index(key)] = action[axis]
...
>>> print larm
[0.1, 0.4, 0.3]

This extracts the features for the left arm from the action vector.

>>> a1 = Action([0.1, 0.4, 0.2])
>>> a2 = Action([0.5, 0.3, 0.5])
>>> print a1 - a2
[-0.4, 0.1, -0.3]

Subtract actions from each other.

>>> print a1 * a2
[0.05, 0.12, 0.1]

Multiplies two actions with each other.

>>> a1 *= a2
>>> print a1
[0.05, 0.12, 0.1]

Multiplies two actions in place.

Attributes

name The name of the MDP primitive.
dtype
nfeatures (int) The number of features.
discretized (bool) Flag indicating whether the features are discretized or not.
min_features (list) The minimum value for each feature.
max_features (list) The minimum value for each feature.
states_per_dim (list) The number of states per dimension.
description (dict) A description of the features.

Methods

DTYPE_FLOAT
DTYPE_INT
DTYPE_OBJECT
decode(_repr) Decodes the state into its original representation.
discretize() Discretizes the state.
dtype
encode() Encodes the state into a human readable representation.
get() Return the feature array.
get_name(features) Retrieves the name of the action.
get_noop_action() Creates a no-op action.
key_to_index(key) Maps internal name to group index.
next()
set(features) Sets the feature array to the given array.
set_description(descr) Set the feature description.
set_discretized([val]) Sets the discretized flag.
set_dtype([value]) Set the feature’s data type.
set_minmax_features(_min, _max) Sets the minimum and maximum value for each feature.
set_nfeatures(n) Set the number of features.
set_states_per_dim(nstates) Sets the number of states per feature.
tolist() Returns the feature array as a list.