ROS development tools, libraries and applications

 

The goal of this project is to support value creation using robots. As a measure for this,

• Realization of ROS application development support tools,
• Realization of environment recognition mechanism suitable for ROS robots,　

are being carried out.

We have already released some software on GitHub.

 

Middleware for ROS applications (GitHub: actor_demo (usage example), ros_actor (for development))

Following pytwb, we developed ros_actor middleware as a mechanism to facilitate ROS application development. This aims to simplify the writing of real-time processing programs by implementing an actor-like programming model on Python.

actor_demo has been created and published a sample application on ros_actor that uses a Turtlebot with an arm to search for, pick, transport, and place cola cans.

ROS application development environment (GitHub: pytwb_tutorial (for use), pytwb (for development))

With the advent of ROS2, the emphasis is placed not only on the development of the robot itself, but also on its utilization. As a result, it is expected that there will be more opportunities for experts in other areas to build ROS applications. In response to this, we are developing a tool that can reduce the development time of a system using ROS, targeting experts in other fields.

in particular, the following uses are assumed.

• Education:
  It is important to enable building realistic robot control with a small burden for educational environment.
• Research:
  Research is conducted using ROS as a tool, not ROS itself, and the people who carry out the research (e.g. machine learning and/or vision areas) do not necessarily have a deep understanding of ROS.
• Prototyping in product planning and solutions:
  Reducing development turnaround time is also important in actual product planning and marketing, especially in prototyping.

 

pytwb has been developed first. The main purpose of this is to support the development of ROS applications using a Python-based behavior tree, and it will be possible to develop ROS applications immediately with a docker environment and VSCode.

The behavior tree presents the skeleton structure of ROS applications, which are becoming increasingly complex, in an easy-to-understand manner, and has the effect of making development easier. In addition, by dividing the program into behavioral units and developing the program, there is an advantage that componentization and reusability are promoted.

One of the purposes of pytwb is to make it possible to standardize ROS application development, like Ruby on Rails for web applications. First, by using the behavior tree, the overall structure of the application can be standardized. In addition, pytwb adds the following features to make behavior trees easier to work with.

• pytwb allows you to write XML behavior tree descriptions, even though Python behavior tree based implementation (py_trees, py_trees_ros),
  • This XML description allows us to divide ROS applications into parts and the relationships between them.
• pytwb allows you to automate description of package dependencies,
• pytwb allows you to skip explicit build steps (direct run) by using docker mechanism with keeping compatibility with the regular building procedure of ROS.

 

The ultimate goal of the project is to be able to develop ROS applications in a stylized way, and pytwb is the starting tool. We are currently improving and adding more tools.

 

Realization of basis for environment recognition function (GitHub: vector_map)

The goal of robot control is to move around in the environment, collect various information from the environment, and take more advanced actions based on collected information. We intend to build a general structure and mechanism for implementing such functionalities. For that purpose, it is important to realize a state space that can properly organize the collected information.

For example, ROS navigation system uses a map based on the occupancy grid format. Recently, with the development of deep learning, in addition to this, it has become possible to collect semantic information such as the shape and type of objects that appear in the environment, as seen in sematic SLAM, and to perform higher-level action planning. In order to aggregate semantic information on environment, it is appropriate to add it to the map, but in the occupancy grid format, the locations of obstacles are represented by point clouds, and information about the structure of the environment, such as walls and areas, is lacking. So, it is difficult to give semantic information to occupancy grid based maps.

Therefore, in this project, with the aim of realizing a map that can add information step by step, the occupancy grid map is further image-processed, and the map is expressed with geometric elements such as line segments and partial curves to create an environmental map. We have developed a mechanism that allows us to retain information about the structure of walls, rooms, corridors, etc. are represented in the form of Python objects with geometric information, making it easy to add further information.

This map mechanism has already been implemented as the “vector_map” library, so in the future we plan to add a mechanism that will add information about the target object and make it available for action planning.

 

test environment

point cloud map by SLAM

vector map

 

 

Full demo (GitHub: pytwb_demo)

We have created a ROS application example using pytwb and vector_map. It is already being used for training at schools and companies because it can easily try out complex behaviors.

View from Robot Camera