Engine DataPacks

DataPacks are simple objects which wrap around arbitrary data structures, like JSON objects or protobuf messages. They provide the necessary abstract interface, which is understood by all components of NRP-Core, while still allowing to pass data in various formats.

A DataPack consists of two parts:

• DataPack ID: which allows to uniquely indentify the object
• DataPack data: this is the data stored by the DataPack, which can be in principle of any type

DataPacks are mainly used by Transceiver functions to relay data between engines. Each engine type is designed to accept only datapacks of a certain type and structure. To discover which datapacks can be processed by each engine, check out the engine's documentation here.

DataPack ID

Every datapack contains a DataPackIdentifier, which uniquely identifies the datapack object and allows for routing of the data between transceiver functions, engine clients and engine servers. A datapack identifier consists of three fields:

• name - name of the datapack. Must be unique.
• type - string representation of the DataPack data type. This field will most probably will be of no concern for the users. It is set and used internally and is not in human-readable form.
• engine name - name of the engine to which the datapack is bound.

These fields can be accessed from the transceiver functions:

print(datapack.name)
print(datapack.type)
print(datapack.engine_name)

DataPack data

DataPack is a template class with a single template parameter, which specifies the type of the data contained by the DataPack. This DataPack data can be in principle of any type. In practice there are some limitations though, since DataPacks, which are C++ objects, must be accessible from TransceiverFunctions, which are written in Python. Therefore the only DataPack data types which can be actually used in NRP-core are those for which Python bindings are provided. These are commented below.

In TransceiverFunctions, the DataPack data can always be accessed using the datapack "data" attribute.

Empty DataPacks

It is possible for a datapack to contain no data. This is useful for example when an Engine is asked for a certain DataPack but it is not able to provide it. In this case, an Engine can return an empty datapack. This type of datapack contains only a datapack identifier and no data.

Attempting to retrieve the data from an empty DataPack will result in an exception. A method "isEmpty" is provided to check whether a DataPack is empty or not before attempting to access its data:

if not datapack.isEmpty():
    # It's safe to get the data
    print(datapack.data)
else:
    # This will raise an exception
    print(datapack.data)

Checking if a DataPack contains the most recent data

It may happen that certain DataPacks that are available to Preprocessing and Transceiver functions will not be updated on every simulation iteration. For example, this will be the case whenever the engines that take part in the simulation run at different frequencies. It may be wasteful to perform processing of the same data multiple times. The DataPack objects have a flag, called isUpdated that allows to check if they contain the most recent data. It will be set to true only on the simulation iteration,on which the DataPack has been created (for example returned from a Preprocessing Function) or received (for example from an Engine). The flag can be accessed from Preprocessing, Transceiver, and Status Functions in the following way:

if datapack.isUpdated():
    # Perform some operations, because the DataPack has just been updated
else:
    # The DataPack is 'stale', i.e. it was updated in one of the previous simulation iterations

Role of DataPacks in TransceiverFunctions

DataPacks are both the input and output of TransceiverFunctions. When a datapack is declared as input of a TF, this datapack is always requested to the corresponding Engine when the latter is synchronized. When a datapack is returned by a TF, it is sent to the corresponding Engine after the TF is executed. For more information about the synchronization model used in NRP-core the reader can refer to these sections:

• Transceiver Functions synchronization
• NRP-core synchronization model

The subsections below elaborate on the details of how to use DataPacks in TFs.

DataPacks as input to transceiver functions

DataPacks can be declared as TransceiverFunction inputs using the dedicated decorator. After that they can be accessed in TFs as input arguments.

# Declare datapack with "datapack_name" name from engine "engine_name" as input using the @EngineDataPack decorator
# The trasceiver function must accept an argument with the same name as "keyword" in the datapack decorator
 
@EngineDataPack(keyword="datapack", id=DataPackIdentifier("datapack_name", "engine_name"))
@TransceiverFunction("engine_name")
def transceiver_function(datapack):
    print(datapack.data)
 
# Multiple input datapacks from different engines can be declared
 
@EngineDataPack(keyword="datapack1", id=DataPackIdentifier("datapack_name1", "engine_name1"))
@EngineDataPack(keyword="datapack2", id=DataPackIdentifier("datapack_name2", "engine_name2"))
@TransceiverFunction("engine_name1")
def transceiver_function(datapack1, datapack2):
    print(datapack1.data)
    print(datapack2.data)

By default, all input DataPacks are passed to Transceiver Functions by value, i.e. the function receives a copy of the original DataPack object. This is a safety measure implemented to prevent accidental modifications of DataPacks available to other Functions and Engines. The policy can be changed by setting the DataPackPassingPolicy configuration parameter.

DataPacks as output of transceiver functions

DataPacks can be returned from the transceiver function.

# NRP-Core expects transceiver functions to always return a list of datapacks
 
@TransceiverFunction("engine_name")
def transceiver_function():
    datapack = JsonDataPack("datapack_name", "engine_name")
 
    return [ datapack ]
 
# Multiple datapacks can be returned
 
@TransceiverFunction("engine_name")
def transceiver_function():
    datapack1 = JsonDataPack("datapack_name1", "engine_name")
    datapack2 = JsonDataPack("datapack_name2", "engine_name")
 
    return [ datapack1, datapack2 ]

Supported DataPack data types

As commented in the section above, DataPacks are both the input and output of TFs. Therefore, a conversion mechanism between C++ and Python is required for each supported DataPack data type. The types currently supported are nlohmann::json and protobuf messages. The subsections below give details of the Python API provided for each of these types.

JsonDataPack

JsonDataPack type wraps around nlohmann::json C++ objects. The Python class wrapping the C++ json object is NlohmannJson, which is stored in JsonDataPack data attribute. NlohmannJson is very flexible and allows to pass most types of data between engines and transceiver functions without writing any additional code, it can contain all basic Python types. NlohmannJson also has partial support for numpy arrays - it's possible to use 1-dimensional arrays of integers and floats.

Importing and creating JsonDataPack

To import JsonDataPack:

from nrp_core.data.nrp_json import JsonDataPack

To create a JsonDataPack object:

datapack = JsonDataPack("datapack_name", "engine_name")

Getting and setting data

Inside transceiver functions the data can be accessed like a python dictionary:

# To set the data
 
datapack = JsonDataPack("datapack_name", "engine_name")
 
datapack.data["null"]   = None
datapack.data["long"]   = 1
datapack.data["double"] = 43.21
datapack.data["string"] = "string"
datapack.data["bool"]   = True
datapack.data["array"]  = [5, 1, 6]
datapack.data["tuple"]  = (1, 2, 3)
datapack.data["object"] = {"key1": "value", "key2": 600}
 
# To retrieve the data
 
print(datapack.data["string"])
print(datapack.data["object"])
 
# JsonDataPack supports dict's __getitem__ and keys methods.
for k in datapack.data.keys():
    print(datapack.data[k])

Numpy arrays:

# Numpy integer arrays
 
datapack.data["numpy_array_int8" ] = np.array([ 1,  2,  3], dtype="int8")
datapack.data["numpy_array_int16"] = np.array([ 4,  5,  6], dtype="int16")
datapack.data["numpy_array_int32"] = np.array([ 7,  8,  9], dtype="int32")
datapack.data["numpy_array_int64"] = np.array([10, 11, 12], dtype="int64")
 
# Numpy unsigned integer arrays
 
datapack.data["numpy_array_uint8" ] = np.array([0,  1,  2], dtype="uint8")
datapack.data["numpy_array_uint16"] = np.array([3,  4,  5], dtype="uint16")
datapack.data["numpy_array_uint32"] = np.array([6,  7,  8], dtype="uint32")
datapack.data["numpy_array_uint64"] = np.array([9, 10, 11], dtype="uint64")
 
# Numpy floating-point arrays
 
datapack.data["numpy_array_float32"] = np.array([0.5, 2.3, 3.55], dtype="float32")
datapack.data["numpy_array_float64"] = np.array([1.5, 2.3, 3.88], dtype="float64")

Inspecting content of JsonDataPack

Printing the content using Python's built-in function str:

str(datapack.data)
str(datapack.data["array"])
str(datapack.data["object"])
 
# Or print it directly:
 
print(datapack.data)

Getting a list of keys:

keys = datapack.data.keys()

Getting length of the object:

length = len(datapack.data)

The above will return number of keys, if data is a JSON object, or number of elements, if it's a JSON array.

Using JsonDataPacks to store JSON arrays

In all the examples above it has been assumed that JsonDataPack is storing a JSON object. Actually the data object can contain either a JSON object, a JSON array or an empty object, it depends on how it is started. After instantiation, it contains an empty object:

datapack = JsonDataPack("example_datapack", "example_engine")
# Returned value is 'null'
datapack.data.json_type()

If data is appended to it, the datapack stores a JSON array:

datapack.data.append(1.55)
# Returned value is 'array'
datapack.data.json_type()

If instead a key is assigned, it stores a JSON object:

datapack = JsonDataPack("example_datapack", "example_engine")
datapack.data['key'] = 1.55
# Returned value is 'object'
datapack.data.json_type()

Please be aware that methods specific to 'object' type for accessing or setting elements will raise an error when used with an 'array' type and otherwise.

Protobuf DataPacks

In contrast with JsonDataPack, which can wrap any nlohmann::json C++ object, a Python wrapper class is generated for each Protobuf definition. For example, for the Camera message listed below (which is used by the Gazebo Engine), a python class GazeboCameraDataPack is generated.

package Gazebo;
 
// Data coming from gazebo camera datapack
message Camera
{
    uint32 imageWidth  = 1;
    uint32 imageHeight = 2;
    uint32 imageDepth  = 3;
    bytes  imageData   = 4;
}

This class contains a data attribute which is of type GazeboCamera and gives access to the wrapped datapack data. The generated Python classes match the original Protobuf Python API as described in the protobuf documentation. There are some known limitations with respect to the original Protobuf Python API which are listed below with references to the protobuf documentation:

1. Well Known Types partially supported - Generic protobuf DataPacks and wrappers for well-known protobuf types.
2. Repeated Message field not supported ref
3. Map field type not supported ref
4. Only basic Enum support. To set or get Enum fields only int can be used. Enum constants can't be accessed from python ref
5. The Message Python wrapper only supports a subset of the methods listed here. These are: 'Clear', 'ClearField', 'HasField', 'IsInitialized' and 'WhichOneof'.

Finally, these Python wrappers are automatically generated in the NRP-core build process for Protobuf message definitions used by Engines shipped with NRPCore. These can be found in the folder nrp-core-msgs/protobuf/engine_proto_defs. See this guide to know how to compile additional messages so they become afterwards available to Engines and TFs.

Generic protobuf DataPacks and wrappers for well-known protobuf types

A set of generic DataPacks is provided with NRP Core. These DataPacks allow to exchange most of the data types available in protobuf, and they can be used to pass arrays of arbitrary size. The data part of the generic DataPacks has the following structure:

message Array*
{
    repeated * array = 1;
    repeated uint32 shape = 2;
}

With the asterisk replacing concrete types.

Another generic DataPack type that is provided with NRP Core is the image DataPack, based on the NrpGenericProto::Image type:

message Image
{
    uint32     height      = 1;
    uint32     width       = 2;
    uint32     step        = 3;
    IMAGE_TYPE type        = 4;
    bool       isBigEndian = 5;
    bytes      data        = 6;
}

The definitions of the protobuf messages used by the DataPacks can be found in the messages repository, in protobuf/engine_proto_defs/nrp_generic.proto.

A subset of wrappers that mimic the (well-known protobuf types) is also available to use. Unfortunately, at the moment it is not possible to use the official wrappers directly, and one must use the wrappers provided with NRP Core. The messages can be found in the messages repository, in protobuf/engine_proto_defs/wrappers.proto.

An example use of the generic DataPacks, as well as the wrappers, can be found in the examples/generic_proto_test directory.

ROS msg datapacks

Similarly to Protobuf datapacks, a Python wrapper class is generated for each ROS msg definition used in NRP-Core. For example, for a message of type Pose from package geometry_msgs, a python class PoseDataPack is generated. This class contains a data attribute which is of type Pose and gives access to the wrapped datapack data. The generated Python bindings can be found under the Python module nrp_core.data.nrp_ros. For example, in the case of the Pose message:

from nrp_core.data.nrp_ros.geometry_msgs import PoseDataPack
p = PoseDataPack("name","engine")
type(p.data) # output is <class 'nrp_core.data.nrp_ros.geometry_msgs.Pose'>

By default Python bindings are generated for all message types in the next ROS packages:

• nrp_ros_msgs (package containing NRP-core ROS message definitions)
• std_msgs
• geometry_msgs
• sensor_msgs

Any message definition contained in these packages can be used in TransceiverFunctions directly. It is also possible to generate Python bindings for messages defined in other ROS packages. For more information can be found here.

NOTE: At this moment there is no Engine implementation which can use ROS datapacks. The only way of interacting with other ROS nodes in NRP-Core is by using a Computational Graph in your experiment instead of Transceiver Functions to rely data between Engines. See this page for more information at this regard.

Implementation details

All concrete datapack classes should be based on the DataPack class. It is a template class, and the single template argument specifies the data structure type, which is going to be held by the class instances.

The DataPack class design is somewhat similar to that of std::unique_ptr. Whenever a datapack object is constructed, it takes ownership of the input data structure. This structure may be then accessed and modified, or the ownership may be released.

The DataPack class inherits from DataPackInterface. This class may also be instantiated, but the object will not carry any data (ie. it's an empty DataPack).

Empty datapacks

A DataPack class is considered empty when its data is released. Every instance of the base class, DataPackInterface, is also considered empty, because there is no data stored in it.

Python interface

In order to be accessible to transceiver functions, a conversion mechanism between C++ and Python must be specified for each DataPack data type. Currently NRP-core provides Python bindings for nlohmann::json and protobuf messages. In case you wished to integrate a different data type, you would have to implement Python bindings for this type and make them available to NRP-core as a Python module.