Last updated: 24 June 2021 By: Tan Guan Yew(guanyewtan) and Zeon Chua (CFZeon)
-
Ubuntu 20.04 LTS (Focal Fossa)
See Ubuntu webpage for installation instructions (version 20.04 release).
These packages were developed and tested using only Ubuntu Linux OS. Although ROS 2 is officially supported on multiple platforms including Windows OS, Ubuntu Linux OS is preferred since there is greater support for packages on Ubuntu platform.
-
Robot Operating System 2 (ROS 2) Foxy Fitzroy Release
Download and installation instructions can be found here.
ℹ️Installing via the Ubuntu terminal and debian packages is the most convenient way to install ROS 2. Omron_TM_ROS2 is currently supported on ROS 2 Foxy only.
This guide assumes you know the basics of using the ROS 2 environment. If you are new to ROS 2, please follow their series of tutorials to get started with using ROS 2.
You will need to complete at least the
Beginner Leveltutorials. -
Python 3
The python scripts and Classes used to run the pick and place program are developed and written in Python 3.8.5.
Information on python Classes can be found here.
-
TMflow v1.82.3400
The TM set up instructions here assumes you are using
TMflow v1.82.3400to set up the listen node (as well as other nodes). The packages are also compatible with TMflow v1.80.5300. Other versions have not been tested. -
Pymodbus
Modbus communication is used alongside the TM drivers to obtain robot information, such as joint positions, and is also capable of controlling many of the TM robot’s features. Documentation for Pymodbus can be found here.
This package assumes a specific hardware configuration in order to work correctly.
-
TM5-900
This will be the TM robot you will be controlling via this package.
ℹ️This package has also been tested to work on the TM12-1300. However, some configurations must be changed to support the proper visualization for the TM12. -
Host machine running ROS 2
This is the machine that communicates with the TM via an ethernet connection. The ROS 2 packages are run on this machine.
-
Robotiq IO Coupling
The packages were designed using the Robotiq IO coupling wrist attachment to control the gripper via digital IO. More information on the IO coupling can be found here.
The diagram below gives an overview of how the communications will be configured:
This package utilises the tmr_ros2 packages for the TM drivers to communicate with the robot.
In order to use this package, your TM must be configured correctly to communicate with the host machine running this ROS 2 package. This configuration will be done via TMflow. This guide assumes that you have basic knowledge on using the TMflow software. More information on the TMflow software can be found in the Software Manual, TMflow SW1.82, 19888-400 RevH.
In TMflow, create a flow project, then choose the Listen node, Vision node, and the Goto node.
|
|
To ensure that the demo program works properly, uncheck the "Start at Initial Position" setting. |
|
ℹ️
|
This step can be skipped if you have imported ROS2_teach.zip provided in the repository files, found under the TM_Export folder. Instructions for importing the zip is located in section 5.8.6 of TMflow Software Manual.
|
-
Go to System → Network setting page and enter the network parameters of the robot
Figure 5. Network Parameters -
Go to Setting → Connection → Ethernet Slave
Select the Data Table Setting button and check the following boxes (make sure the Ethernet Slave is DISABLED or you will be unable to change the settings):
-
✓ Robot_Error
-
✓ Project_Run
-
✓ Project_Pause
-
✓ Safeguard_A
-
✓ ESTOP
-
✓ Camera_Light
-
✓ Error_Code
-
✓ Joint_Angle
-
✓ Coord_Robot_Flange
-
✓ Coord_Robot_Tool
-
✓ TCP_Force
-
✓ TCP_Force3D
-
✓ TCP_Speed
-
✓ TCP_Speed3D
-
✓ Joint_Speed
-
✓ Joint_Torque
-
✓ Project_Speed
-
✓ MA_Mode
-
✓ Robot Light
-
✓ Ctrl_DO0~DO7
-
✓ Ctrl_DI0~DI7
-
✓ Ctrl_AO0
-
✓ Ctrl_AI0~AI1
-
✓ END_DO0~DO3
-
✓ END_DI0~DI2
-
✓ END_AI0
-
-
Set the Data Format to BINARY and press save.
The host machine is used to run the python scripts which use both the TM Drivers and the ROS 2 packages to communicate with the TM robot. Ensure that your machine meets the requirements in Section 1.1, “Prerequisites”.
Ensure that the ip address of the host machine and the TM robot(Section 1.3.2, “TM System/Network Setup”) have the same subnet. Instructions on how to change the static ip address in Ubuntu 20.04 can be found here
Once you have your network set up correctly, you need to configure this ROS 2 package to work correctly in your host machine.
First, make sure you have installed ROS 2 as described in Section 1.1, “Prerequisites”.
-
Clone this repository to a directory of your choice with:
cd <directory> git clone https://github.com/guanyewtan/Omron_TM_ROS2
-
Enter the folder with:
cd Omron_TM_ROS2
-
Build all package with:
colcon build --symlink-install
Depending on your machine, this can take a while to build. If you receive a warning saying "no such command", follow the intructions here.
|
ℹ️
|
You might need to install some missing packages if you didn’t already have them. They can be installed with sudo apt install ros-foxy-control-msgs.
|
-
Enter your ROS 2 workspace and source the ROS2 environment:
source /opt/ros/foxy/setup.bash cd <workspace> source ./install/setup.bash -
Ensure that TM Robot’s operating software (TMflow) system/network settings have been set and the Listen node is running (run the project above)
-
Run the driver to maintain the connection with TM Robot:
ros2 run tm_driver tm_driver <robot_ip_address>Example:
ros2 run tm_driver tm_driver 192.168.2.10, if the <robot_ip_address> is 192.168.2.10Now, the user can use a new terminal to run each ROS node or command, but don’t forget to source the correct setup shell files after starting a new terminal!
The TM driver is required to be running as long as a connection to the listen node of the TMflow program is required.
For more information on the TM Drivers, click here.
An overview of this package architecture is summarised in the diagram below:
Users can establish a socket TCPlistener in the listen node to connect to external device and communicate based on the packet format.
All features available in TM ROBOT Function can be operated in the listen node. For more information on the listen node, please refer to page 181 of the TM Expression Editor and Listen Node Reference Guide.
The TM Driver utilises TMSCT and TMSTA communication packages to send external scripts and obtain the status or properties of the TM respectively. Below is an example of how the TM Driver uses these 2 communication packages to communicate through the Socket TCPlistener:
-
TM Driver sends a PTP (point-to-point) movement command via a ROS2 service client using TMSCT packages to the TM Robot. When the command has been successfully sent, an acknowledgement is sent back to the host machine.
-
A queue tag is sent via TMSCT communication packages and its status monitored using TMSTA, to check if a motion command has been completed.
Users can use Modbus Client to read or write the parameters and save them in the robot register, such as position, posture and IO status. Users can program with the obtained parameters or monitor the status of robot. TM Robot provides two protocol versions of Modbus: Modbus TCP and Modbus RTU for users to obtain data from the external Modbus device or robot register. Modbus TCP is used for these packages.
There are limitations to the capabilities of the TM Drivers, which is why Modbus is being used to send and receive information unobtainable by the drivers, such as getting the coordinates of the current base in the project flow or starting the project from outside the listen node.
The pymodbus libraries are used to communicate with the modbus servers.
The RViz package allows a 3D model TM robot to be displayed in a separate window for real time visualisation.
This package uses the joint states generated from the TM driver as well as the robot description publisher to generate the model and display it in its current position.
To understand how RViz Visualisation is structured with the entire ROS package and communicates with LD, see Section 2.1, “Architecture”.
RViz Visualisation has three nodes, they are summarised as below:
| Node name | Description |
|---|---|
tm_driver |
This node is responsible for publishing the joint states that the RViz program subscribes to. Using this information, it updates the position of the TM shown on RViz. |
robot_description |
This node is responsible for pubishing the .urdf information that the RViz program uses to display the model of the robot, as well as know the transform of each component of the robot relative the another point. |
static_transform_publisher |
This node is responsible for publishing a transform which sets the base of the 3D model to the zero coordinate. |
pp_marker |
This node is responsible for publishing a transform and marker that represents an object picked up by the grippers. |
tcp_publisher |
This node is responsible for publishing a transform to the flange, representing the location of the TCP that is set in TMFlow |
The pickplace package allows the user to create a pick and place program, requiring only a one time setup. The pickplace program will then continuously run a vision guided pick and place operation.
|
ℹ️
|
This implementation was designed using a TM Landmark attached to the object to be picked, and another TM landmark to find the place location. However, it is possible for the TM vision to locate other visual features on objects, depending on how the vision job is trained in TMFlow. |
These 2 landmarks act as the vision base for the pick and place, and the vision job takes into account the tilt and rotation of the TM landmark (so long as it can be seen clearly by the camera)
There are 2 stages to the program: a setup phase, where the user sets the location of TM to view the pick and place landmarks as well as the pick and place locations, and an execution phase, where the pick and place operation will run based on the coordinates set in the setup phase.
The teach_setup.py script runs through a sequence of instructions for the user to record the pick and place positions of an object, as well as the positions to view their respective landmarks. The following diagram shows the flow of the setup program:
-
In one terminal, run
server.launch.pyto ensure that the teach_setup program can communicate with the TM robot withros2 launch pickplace server.launch.py robot_ip:=<robot_ip_address>
-
In another terminal, run the program to initiate the setup process, replacing
robot_ip_addresswith the ip address of the TM robot.ros2 run pickplace teach_setup <robot_ip_address>
ℹ️Remember to source the ROS2 packages again in the new terminal or you might not be able to run the setup program properly. -
Move the robot to an initial position to start with on program launch, then press ENTER
-
Enter the name of the vision base that will be created in the TMflow program by the vison job.
|
ℹ️
|
The vision base name is the name of the vision job with vision_ appended to the front. For example, if the vision job name is myvisionjob, the vision base name will be vision_myvisionjob.
|
-
Move the TM arm to the landmark viewing position for picking the object.
-
Press the play button on the robot stick. You should hear the robot beep 3 times to signal the start of the TMflow project.
-
Hit ENTER to start the vision job.
-
Wait for the vision job to complete, then press the stop button on the robot stick. The robot should beep once.
ℹ️It is advised to open up the vision job to check if the landmark can be detected, or this might cause problems with detection during the execution process. 
-
Move the TM arm to the pick position.
⚠️ Ensure that the object remains in the SAME POSITION as it was during the vision job. -
Hit ENTER to close the grippers and record the position, then move the TM arm to the landmark viewing position for placing the object.
-
Press the play button on the robot stick. The robot should beep 3 times.
-
Hit ENTER to start the vision job.
ℹ️It is advised to open up the vision job to check if the landmark can be detected, or this might cause problems with detection during the execution process. -
When the vision job is done, press the stop button on the robot stick. The robot should beep once.
-
Move the TM arm to the place position and hit ENTER, which releases the grippers and records the position.
-
A JSON file will be generated containing the coordinates of the landmark viewing positions, the pick and lace positions as well as the name of the vision job, to be used in the execution stage.
Should you wish to configure your own end effector instead of the Robotiq 2F-85 gripper, there are a few steps to be followed.
-
Return to TMFlow
-
Go to Setting → TCP Setting → Manual input parameters of TCP
-
Set the appropriate position of your TCP relative to the robot flange
-
Click Apply
Your TCP is now configured for use in this package.
The pickplace_program.py script takes the coordinates obtained from the setup stage and implements a continuous pick and place program, with an RViz terminal showing the real time pose of a 3D model of the TM robot. The flow of the program is shown below:
-
. Run the launch file to execute the program, replacing
robot_ip_addresswith the ip address of the TM robot.ros2 launch pickplace pickplace.launch.py robot_ip:=<robot_ip_address>
This will run the pick & place program as well as the RViz window for the 3D model visualisation.
|
Caution
|
The Script class is a class that handles the scripts to be sent to the TCP socket listener in the TM machine via the TM driver. It does not contain the pickplace program.
|
The movement of the TM arm is controlled by the Move class. The set_position method takes in x, y, z, roll, pitch and yaw, and sends it to the listen node in the TMflow program via a service client from the TM Driver.
|
ℹ️
|
Queue tags and status queries were also used to ensure that no other command executes simultaneously until the motion is complete. For more information, please refer to section 8.1 of the TM Expression Editor and Listen Node Reference Guide under Robot Motion Functions. |
This class handles all the transforms between different frames (e.g. coordinates of the pick position relative to the robot base or vision base). The TF tree is shown below:
|
ℹ️
|
safe_pick and safe_place transforms are the coordinates of the pick and place point with a -0.1m offset in the z-axis |
The Script class uses the external scripts function of the TM Driver to run external commands.
|
ℹ️
|
The script class is capable of sending instructions to run TM movement commands, however the TM SetPosition service client from the tmr_ros2 package is preferred as the format of the parameters are easier to configure and use. |
The modbus class handles modbus communication between the client (TM machine) and server (user’s computer). It allows for asynchronous control of many features of the TM robot.
|
ℹ️
|
The TM Driver provides services to open and close the IO gripper. However, modbus control of the IO is preferred as it does not require the TM Driver to be running concurrently with the program, as running the TM Driver to control the IO would require the TMflow program to be in the listen node, which prevents the TM robot from being able to be manually moved to the setup locations during the setup stage. |
The IO class creates a client to the modbus server to open and close the Robotiq IO gripper. It serves to simplify the API for interfacing with the TM robot.