Sensors¶
Definition of all of the sensor information
Classes:
|
Returns True if the agent has been abused. |
|
|
|
For the CupGame task, returns which cup the ball is underneath. |
|
|
|
Returns true if the agent is colliding with anything (including the ground). |
|
|
|
|
|
|
|
Base class for a sensor |
|
Inertial Measurement Unit sensor. |
|
Returns the state of the |
|
Gets the location of the agent in the world. |
|
|
|
Gets the forward, right, and up vector for the agent. |
|
For each joint on the |
|
Captures agent’s view. |
|
Returns distances to nearest collisions in the directions specified by the parameters. |
|
Gets the position of each bone in a skeletal mesh as a quaternion. |
|
Gets the rotation of the agent in the world. |
|
A class for new sensors and their parameters, to be used for adding new sensors. |
|
Given a sensor definition, constructs the appropriate HolodeckSensor object. |
|
Returns the x, y, and z velocity of the agent. |
|
Captures what the viewport is seeing. |
|
Returns any numeric value from the world corresponding to a given key. |
-
class
holodeck.sensors.
AbuseSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Returns True if the agent has been abused. Abuse is calculated differently for different agents. The Sphere and Hand agent cannot be abused. The Uav, Android, and Turtle agents can be abused by experiencing high levels of acceleration. The Uav is abused when its blades collide with another object, and the Turtle agent is abused when it’s flipped over.
Configuration
AccelerationLimit
: Maximum acceleration the agent can endure before being considered abused. The default depends on the agent, usually around 300 m/s^2.
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
class
holodeck.sensors.
AvoidTask
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
BallLocationSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ For the CupGame task, returns which cup the ball is underneath.
The cups are numbered 0-2, from the agents perspective, left to right. As soon as a swap begins, the number returned by this sensor is updated to the balls new position after the swap ends.
Only works in the CupGame world.
Attributes:
dtype
The type of data in the sensor
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
CleanUpTask
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
Methods:
start_task
(num_trash[, use_table])Spawn trash around the trash can.
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
start_task
(num_trash, use_table=False)¶ Spawn trash around the trash can. Do not call if the config file contains a clean up task configuration block.
- Parameters
num_trash (
int
) – Amount of trash to spawnuse_table (
bool
, optional) – If True a table will spawn next to the trash can, all trash will be on the table, and the trash can lid will be absent. This makes the task significantly easier. If False, all trash will spawn on the ground. Defaults to False.
-
property
-
class
holodeck.sensors.
CollisionSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Returns true if the agent is colliding with anything (including the ground).
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
CupGameTask
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
Methods:
start_game
(num_shuffles[, speed, seed])Start the cup game and set its configuration.
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
start_game
(num_shuffles, speed=3, seed=None)¶ Start the cup game and set its configuration. Do not call if the config file contains a cup task configuration block, as it will override the configuration and cause undefined behavior.
- Parameters
num_shuffles (
int
) – Number of shufflesspeed (
int
) – Speed of the shuffle. Works best between 1-10seed (
int
) – Seed to rotate the cups the same way every time. If none is given, a seed will not be used.
-
property
-
class
holodeck.sensors.
DistanceTask
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
FollowTask
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
HolodeckSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Base class for a sensor
- Parameters
client (
HolodeckClient
) – Client attached to a sensoragent_name (
str
) – Name of the parent agentagent_type (
str
) – Type of the parent agentname (
str
) – Name of the sensorconfig (
dict
) – Configuration dictionary to pass to the engine
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
sensor_data
Get the sensor data buffer
Methods:
rotate
(rotation)Rotate the sensor.
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
rotate
(rotation)¶ Rotate the sensor. It will be applied in approximately three ticks.
step()
ortick()
.)This will not persist after a call to reset(). If you want a persistent rotation for a sensor, specify it in your scenario configuration.
- Parameters
rotation (
list
offloat
) – rotation for sensor (see Rotations).
-
property
sensor_data
¶ Get the sensor data buffer
- Returns
Current sensor data
- Return type
np.ndarray
of sizeself.data_shape
-
class
holodeck.sensors.
IMUSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Inertial Measurement Unit sensor.
Returns a 2D numpy array of
::`
- [ [acceleration_x, acceleration_y, acceleration_z],
[velocity_roll, velocity_pitch, velocity_yaw] ]
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
class
holodeck.sensors.
JointRotationSensor
(client, agent_name, agent_type, name='RGBCamera', config=None)¶ Returns the state of the
AndroidAgent
’s or theHandAgent
’s joints.See Android Joints or HandAgent Joints for the indexes into this vector.
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
LocationSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Gets the location of the agent in the world.
Returns coordinates in
[x, y, z]
format (see Coordinate System)Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
LocationTask
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
OrientationSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Gets the forward, right, and up vector for the agent. Returns a 2D numpy array of
[ [forward_x, forward_y, forward_z], [right_x, right_y, right_z ], [up_x, up_y, up_z ] ]
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
PressureSensor
(client, agent_name, agent_type, name='RGBCamera', config=None)¶ For each joint on the
AndroidAgent
or theHandAgent
, returns the pressure on the joint.For each joint, returns
[x_loc, y_loc, z_loc, force]
, in the order the joints are listed in Android Joints or HandAgent Joints.Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
RGBCamera
(client, agent_name, agent_type, name='RGBCamera', config=None)¶ Captures agent’s view.
The default capture resolution is 256x256x256x4, corresponding to the RGBA channels. The resolution can be increased, but will significantly impact performance.
Configuration
The
configuration
block (see Configuration Block) accepts the following options:CaptureWidth
: Width of captured imageCaptureHeight
: Height of captured image
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
Methods:
set_ticks_per_capture
(ticks_per_capture)Sets this RGBCamera to capture a new frame every ticks_per_capture.
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
set_ticks_per_capture
(ticks_per_capture)¶ Sets this RGBCamera to capture a new frame every ticks_per_capture.
The sensor’s image will remain unchanged between captures.
This method must be called after every call to env.reset.
- Parameters
ticks_per_capture (
int
) – The amount of ticks to wait between camera captures.
-
class
holodeck.sensors.
RangeFinderSensor
(client, agent_name, agent_type, name='RangeFinderSensor', config=None)¶ Returns distances to nearest collisions in the directions specified by the parameters. For example, if an agent had two range sensors at different angles with 24 lasers each, the LaserDebug traces would look something like this:
Configuration
The
configuration
block (see Configuration Block) accepts the following options:LaserMaxDistance
: Max Distance in meters of RangeFinder. (default 10)LaserCount
: Number of lasers in sensor. (default 1)LaserAngle
: Angle of lasers from origin. Measured in degrees. Positive angles point up. (default 0)LaserDebug
: Show debug traces. (default false)
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
class
holodeck.sensors.
RelativeSkeletalPositionSensor
(client, agent_name, agent_type, name='RGBCamera', config=None)¶ Gets the position of each bone in a skeletal mesh as a quaternion.
Returns a numpy array with four entries for each bone (see AndroidAgent Bones or HandAgent Bones for the order used)
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
RotationSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Gets the rotation of the agent in the world.
Returns
[roll, pitch, yaw]
(see Rotations)Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
SensorDefinition
(agent_name, agent_type, sensor_name, sensor_type, socket='', location=(0, 0, 0), rotation=(0, 0, 0), config=None, existing=False)¶ A class for new sensors and their parameters, to be used for adding new sensors.
- Parameters
agent_name (
str
) – The name of the parent agent.agent_type (
str
) – The type of the parent agentsensor_name (
str
) – The name of the sensor.sensor_type (
str
orHolodeckSensor
) – The type of the sensor.socket (
str
, optional) – The name of the socket to attach sensor to.location (Tuple of
float
, optional) –[x, y, z]
coordinates to place sensor relative to agent (or socket) (see Coordinate System).rotation (Tuple of
float
, optional) –[roll, pitch, yaw]
to rotate sensor relative to agent (see Rotations)config (
dict
) – Configuration dictionary for the sensor, to pass to engine
Methods:
get_config_json_string
()Gets the configuration dictionary as a string ready for transport
-
get_config_json_string
()¶ Gets the configuration dictionary as a string ready for transport
- Returns
The configuration as an escaped json string
- Return type
(
str
)
-
class
holodeck.sensors.
SensorFactory
¶ Given a sensor definition, constructs the appropriate HolodeckSensor object.
Methods:
build_sensor
(client, sensor_def)Constructs a given sensor associated with client
-
static
build_sensor
(client, sensor_def)¶ Constructs a given sensor associated with client
- Parameters
client (
str
) – Name of the agent this sensor is attached tosensor_def (
SensorDefinition
) – Sensor definition to construct
Returns:
-
static
-
class
holodeck.sensors.
VelocitySensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Returns the x, y, and z velocity of the agent.
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property
-
class
holodeck.sensors.
ViewportCapture
(client, agent_name, agent_type, name='ViewportCapture', config=None)¶ Captures what the viewport is seeing.
The ViewportCapture is faster than the RGB camera, but there can only be one camera and it must capture what the viewport is capturing. If performance is critical, consider this camera instead of the RGBCamera.
It may be useful to position the camera with
teleport_camera()
.Configuration
The
configuration
block (see Configuration Block) accepts the following options:CaptureWidth
: Width of captured imageCaptureHeight
: Height of captured image
THESE DIMENSIONS MUST MATCH THE VIEWPORT DIMENSTIONS
If you have configured the size of the viewport (
window_height/width
), you must make sure thatCaptureWidth/Height
of this configuration block is set to the same dimensions.The default resolution is
1280x720
, matching the default Viewport resolution.Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
class
holodeck.sensors.
WorldNumSensor
(client, agent_name=None, agent_type=None, name='DefaultSensor', config=None)¶ Returns any numeric value from the world corresponding to a given key. This is world specific.
Attributes:
data_shape
The shape of the sensor data
dtype
The type of data in the sensor
-
property
data_shape
¶ The shape of the sensor data
- Returns
Sensor data shape
- Return type
tuple
-
property
dtype
¶ The type of data in the sensor
- Returns
Type of sensor data
- Return type
numpy dtype
-
property