Robotic Creatures

My last post was about visualization of ROS Markers with the OpenSceneGraph rendering library. I have now added the interactive part, thus allowing OSG-based applications to render Interactive Markers, and users to interact with them. There are still things to do, like returning feedback, or support for menus. The code, still experimental, is under the visualization_osg stack in our google code repository. More to come!

These days I’m trying to implement rendering of ROS Interactive Markers in OpenSceneGraph, so that we can start using them inside UWSim. Fortunately I’m able to reuse many of the code already implemented in rviz. I have first focused on the rendering of visualization_msgs/Markers:  had to replace the calls to Ogre by the equivalents in OSG, and remove any dependency with rviz-specific classes. I guess things will become a bit more difficult on the “Interactive” part :-S

Below is a snapshot of UWSim listening to the output of the basic_controls tutorial. I have started a new experimental stack visualization_osg in our uji-ros-pkg repository.

Cloud robotics is getting more and more attention recently. See for example the Google initiative presented at I/O 2011, or the RoboEarth project funded by the European Comission. The idea is to let robots share information on the cloud with the goal of generating a global knowledge base and ultimately accelerating robotic development.

We recently presented rostweet, a ROS package that lets your ROS-based robots post text to a twitter account. We now extend it and add support for posting pictures, by using the recent post_with_media twitter API. This enables the sharing of images among ROS-based robots through twitter, which could have interesting applications in ‘social’ object recognition, learning, or just communication among robots and between robots and humans.

This is how it works: by running rostweet, a ROS service is created that your node can call for posting a tweet with some text (string) and an optional image (sensor_msgs/Image). In addition, incoming tweets are published on a topic as they arrive, also in the format of string + sensor_msgs/Image. Just create a twitter account for your robot and start following other robots (or humans!). For now, rostweet only supports the image media type, although support for other types of media can be easily added.

Please check our first post for installation instructions, although there are some changes in this last version:

• Posting a tweet is now done though a service call. An example node called ‘post’ has been included in the package. Use it as rosrun rostweet post "<text>" [<path_to_picture>]
• The rostweet_msgs/IncomingTweet message now includes also an image. To see just the text of the incoming tweets, do rostopic echo /rostweet/incomingTweet/tweet

Hi, for those who may need it, this is a simple C++ class that loads a point cloud file (PCD format of the Point Clouds Library PCL) and creates an OpenSceneGraph geode that you can easily add to your OSG scene.

#ifndef OSGPCDLOADER_H_
#define OSGPCDLOADER_H_

#include <pcl/io/pcd_io.h>
#include <pcl/point_types.h>

#include <osg/Geometry>
#include <osg/Geode>

class osgPCDLoader {
public:
  pcl::PointCloud<pcl::PointXYZRGB> cloud;
  osg::ref_ptr<osg::Geode> geode;

  osgPCDLoader(std::string pcd_file) {
   if (pcl::io::loadPCDFile<pcl::PointXYZRGB> (pcd_file, cloud) == -1)  // load the file
   {
     std::cerr << "Couldn't read file " << pcd_file << std::endl;
   } else {
     std::cout << "Loaded " << cloud.width * cloud.height << " data points from " << pcd_file << std::endl;

     geode=osg::ref_ptr<osg::Geode>(new osg::Geode());
     osg::ref_ptr<osg::Geometry> geometry (new osg::Geometry());

     osg::ref_ptr<osg::Vec3Array> vertices (new osg::Vec3Array());
     osg::ref_ptr<osg::Vec4Array> colors (new osg::Vec4Array());

     for (int i=0; i<cloud.points.size(); i++) {
       vertices->push_back (osg::Vec3 (cloud.points[i].x, cloud.points[i].y, cloud.points[i].z));
       uint32_t rgb_val_;
       memcpy(&rgb_val_, &(cloud.points[i].rgb), sizeof(uint32_t));

       uint32_t red,green,blue;
       blue=rgb_val_ & 0x000000ff;
       rgb_val_ = rgb_val_ >> 8;
       green=rgb_val_ & 0x000000ff;
       rgb_val_ = rgb_val_ >> 8;
       red=rgb_val_ & 0x000000ff;

       colors->push_back (osg::Vec4f ((float)red/255.0f, (float)green/255.0f, (float)blue/255.0f,1.0f));
     }

     geometry->setVertexArray (vertices.get());
     geometry->setColorArray (colors.get());
     geometry->setColorBinding(osg::Geometry::BIND_PER_VERTEX);

     geometry->addPrimitiveSet(new osg::DrawArrays(osg::PrimitiveSet::POINTS,0,vertices->size()));

     geode->addDrawable (geometry.get());
     osg::StateSet* state = geometry->getOrCreateStateSet();
     state->setMode( GL_LIGHTING,osg::StateAttribute::OFF);
    }
  }

  ~osgPCDLoader(){}

};
#endif

This is an example of a point cloud loaded into UWSim:

Just for fun & profit I  have attached a head-massaging ‘tool’ to a mechanical structure with one servo-controlled joint. The servo is plugged to an arduino board that is integrated in ROS via the rosserial stack. On the other hand, a mobile phone running android executes a very simple app consisting of just a SeekBar. The value of the seekbar is published to the ROS network thanks to rosjava, and therefore the actuated joint is moved accordingly. The perfect gift for these christmas

The following video shows an Android application sending position references to a simulated robotic arm by using rosjava. Rosjava is a java port of ROS (Robot Operating System). It allows to run ROS nodes inside a java application, e.g. typical Android apps. The application running on the phone just displays 5 SeekBars, each one corresponding to one joint of a robotic arm (CSIP ARM5E in this case). Each time a bar thumb is moved, a ROS JointState message is published by the ROS node running on the phone, and received by other nodes that are subscribed to the topic. In this case, our underwater simulator UWSim was receiving the JointState messages and updating the simulated arm accordingly. The same could be done with the real arm that accepts the same JointState interface. More to come!

Rostweet is a twitter client for ROS. It allows your robots to send tweets to its followers, and to receive tweets from its following contacts. Rostweet can be applied in many different ways, these are some examples:

• Communicating progress. Robots running ROS can use rostweet for reporting feedback: task progress, failure detection, etc.
• Sharing new knowledge. After acquiring new information, a robot can use rostweet in order to share it with its followers, e.g. how to grasp an object, a new environment map, etc.
• Acquiring new knowledge. By listening to the others’ tweets, a new source of incoming information is created and can be used for enlarging the robot knowledge.
• Just fun! Let your robot read your tweets, etc.

In general, rostweet can be useful for instantaneous and decentralized robot-robot, robot-human and human-robot sharing of information.

Rostweet is delivered as a ROS stack. It can be downloaded from the uji-ros-pkg google code project repository:

svn co http://uji-ros-pkg.googlecode.com/svn/trunk/rostweet

After downloading and placing it in a folder inside your ROS_PACKAGE_PATH, it should compile with:

rosmake rostweet

and run with:

rosrun rostweet rostweet

This will ask for a twitter username and password and start the node afterwards (by running the node for the first time, you implicitly accept rostweet as a twitter application in your account, and allow it to read your timeline and publish tweets). Once the node is running, other nodes can subscribe to incoming tweets at /rostweet/incomingTweet (of type rostweet_msgs/IncomingTweet), and publish tweets by calling to the service /rostweet/postTweet (of type rostweet_msgs/postTweet). There is an example node for posting e.g.:

View incoming tweets:

rostopic echo /rostweet/incomingTweet/tweet

Send a tweet:

rosrun rostweet post "Hello world from rostweet"

Without any parameters, each time rostweet is executed, it will ask for username, password, and will get the OAuth tokens. If you want to skip some of these steps, you can set them with the following parameters via rosparam:

After the first execution, the username and token keys are stored in a file inside the launch/ folder in YAML format. With it you can easily write a launch file in order to set the parameters automatically:

<launch>
<node name="rostweet" pkg="rostweet" type="rostweet" output="screen">
<rosparam command="load" file="$(find rostweet)/launch/paramFile.yaml" />
</node>
</launch>

Please check also our more recent post about tweeting with pictures.
For support, please post to ROS-Answers.