tutorial
reference
bugs
CVS
BBS

TekkotsuMon Tutorial

TekkotsuMon is the collection of tools which provide control and visualization/monitoring of the robot's run time state.  There are three levels to the interface: local/physical, console, and GUI.  You can read more about the UI philosophy in the ControlBase documentation.  (ControlBase is the base class for all of the menu items you will see below.)

Available controls:

Click here to see a list of the controls available by default at run time.

Level 1: Physical Access

With the Aibo series of robots, you need nothing more than the robot itself to interact with the control system.  Any time the emergency stop is activated (look for the pulsing tail LEDs) you will be able to access the Controller through the head buttons on an ERS-210, or the tail buttons on an ERS-220. Robots which lack any direct human interface are unfortunately unable to use this level of the system.


 ERS-210: ERS-220: ERS-7:
Next Item: Front Head Button Left Tail Button Front Back Button
Previous Item: Rear Head Button Right Tail Button Rear Back Button
Select Item: Chin Button Center Tail Button Center Back Button
Cancel ("Return"): Back Button Main Back Button
 Head Button
Request Text Input Prompt: Press "Next" and "Previous" buttons at the same time.

The Aibo's system console has a limitation that when input is being read, all execution in that process halts until the read has completed.  Thus, you need to request a prompt when you wish to enter data on the system console.  The Tekkotsu console (see below) does not have this limitation, and thus will ignore "Text Input Prompt" requests if the Tekkotsu console is connected.

Feedback is provided through sound and the LEDs.  The 210 will display a "number", whereas you can simply count the number of LEDs on the heads of the 220 and 7.

Level 2: Console

For development, a console is indispensable - debugging would be near impossible without it.  Of course, this requires a computer, wireless ethernet hardware, and implies physical separation from the robot, which is why it's important to allow users to do basic operations in Level 1 without all of those requirements.  At this point, note that a stand to elevate from the ground can be very useful in development to prevent unexpected (and costly) motions which could cause a robot to fall off of the table.

There are several different consoles, each accessible by telneting to a different port.

  • System console - direct terminal console (or port 59000 on Aibo)
    • All output from system calls, printf, cout, and cerr goes here.
    • Output from sout (below) will be redirected here if it is not connected.
    • Output is non-blocking, so if a crash occurs, the last few lines of output may not be displayed.
    • Reading from this console (cin) is blocking - your code will freeze until the user hits return, and there is no way (that we know) to check if input is waiting. (hence the need for the user to specify when they are ready to input data using the buttons listed above)
  • Tekkotsu standard out (sout)- port 10001
    • If you connect to this port, all tekkotsu-generated output will be sent here instead of the system console. This can help you separate framework messages from system messages, or your own debugging messages. (if you are using cout)
    • Output from serr is sent here if there is no other connection.
    • Output is non-blocking.
    • Input from this port is sent to the controller.  If there is a GUI active, anything you type will be broadcast as a TextMsgEvent.  If there is no GUI active, the input will be processed by the controller (commands listed below). This behavior can be modified via tekkotsu.cfg
  • Tekkotsu standard error (serr) - port 10002
    • Output is blocking.  Thus, if you send something here, your code will stop until the message is sent.  This is very handy for debugging so you can tell what it was doing right before a crash, but is also rather slow, so it should be used sparingly.
      • NOTE: It appears this is not completely blocking.  There appears to be some system buffering, so that a few lines of output may still be lost.
    • If this port is unconnected, it will be redirected to sout, but then no claims about blocking can be made (sout is responsible for its transmission).
    • There is no input from this port.

When sout is being used as the main display for the Controller, (i.e. when there is no open GUI) the following commands are available: (from the Controller documentation)

  • '!refresh' - redisplays the current control (handy on first connecting, or when other output has scrolled it off the screen)
  • '!reset' - return to the root control
  • '!next' - calls doNextItem() of the current control
  • '!prev' - calls doPrevItem() of the current control
  • '!select' - calls doSelect() of the current control
  • '!cancel' - calls doCancel() of the current control
  • '!msg text' - broadcasts text as a TextMsgEvent
  • '!root text' - calls takeInput(text) on the root control
  • '!hello' - responds with 'hello\ncount\n' where count is the number of times '!hello' has been sent.  Good for detecting first connection after boot vs. a reconnect.
  • '!hilight [n1 [n2 [...]]]' - hilights zero, one, or more items in the menu
  • '!input text' - calls takeInput(text) on the currently hilighted control(s)
  • '!set section.key = value' - will be sent to Config::setValue(section,key,value)
  • any text not beginning with ! - sent to takeInput() of the current control

By default, most controls provide a menu interface.  Typing the number of an item or a unique prefix will select it.

Level 3: GUI

To ensure platform portability, we have written our GUI tools using Java, and have some legacy code through MATLAB which may be of interest

Both the MATLAB and Java versions share network code.  Each network connection to the Aibo is maintained by a Java object, which is then either used by a MATLAB interface or a Java interface.  In this way, there is no code duplication.

Available Java tools are listed below.

Java Tools

Setting up TekkotsuMon

To use the Java tools, you will need Sun's Java Software Development Kit 1.4 or higher. The machine you are running on needs to be able to access the Aibo's IP address. Beware firewalls at NAT issues!

If you had Java when you compiled the framework, make should have already built the tools.  Change directory to

  <tekkotsu-root>/tools/mon

Type make now if the tools are not already built. (look for .class files in org/tekkotsu/mon subdirectory).
To launch the main GUI, type:

   ./ControllerGUI <aibo-ip-address>

This should bring up the ControllerGUI window similar to the one shown in the next screenshot.

The Controller

The Controller is the main interface to edit the robot's run time state.  It provides all of the capabilities of the console - messages typed in its input field are interpreted exactly as they are on the console.

Double clicking menu items will open them.  Items with arrowheads on the right signify submenus.  The menu system contains a number of useful tools for editing parameters and producing various status reports and logs.

The chasing arrows in the lower right will force a reconnect - this is handy if the robot is forceably rebooted and the Controller doesn't notice immediately.

The protocol used in communication with the ControllerGUI is defined in Controller's documentation.  This same protocol can be used manually to send commands to port 10001 if the GUI isn't connected.

Vision Monitor

You can stream video from the robot in real time and monitor the performance of the color segmentation.  You can also record these streams to image sequences on the desktop for later processing.

The Raw Vision window also lets you select what color space to view.

The network tends to be the main bottleneck - the FPS is relative to frames received by the GUI, not necessarily the FPS the robot is processing on its end.

You can modify compression, resolution, and channel options from the tekkotsu.cfg file (<project>/ms/config/tekkotsu.cfg)  However, you can also change these options at run time using the !set command (see example here)

Watchable Memory GUIWatchable Memory

Using a bit of C++ overloading, we can monitor all changes to primitive data types without changes to existing code.  You can then see the current, real-time values of relevant variables in a window as shown here, or have other sections of code listen for value changed events so they can update themselves as well.  Note in the screen shot the recursive structure of embedded registries to keep related variables together.  Custom GUIs can be created to visualize more complex data types.

Unfortunately, the screen shot isn't nearly as exciting as the real thing, since the whole point is that the values automatically update.  To demo this, the ChaseBallBehavior (under the Mode Switch menu) features two variables that store the last known veritcal and horizontal position of the ball.  Launch the ChaseBallBehavior and then launch "View WMVars" under the TekkotsuMon menu. (you may need to press "refresh" within the WMVar Viewer to see new variables)

Walk Remote Control

This interface allows you to walk the Aibo around, and is overloaded to drive a wheeled robot as well - combined with the vision monitors, it's everything you need for basic telepresence.  Using the right mouse button in the drag area will temporarily reverse the Rotate/Strafe setting.

Head Point Remote Control

This interface allows you to control the Aibo's head.  It is very similar to the interface used for the Walk GUI.

Future version may include additional HeadPointerMC features.

Aibo3D

Using 3D models released by Sony, we have build a tool which will allow you to visualize the Aibo's current posture, or grab joints and move them around in 3D.

This tool requires java3D to be installed.  (Available for Linux and Windows)


Emergency Stop Control

This interface allows you to turn the emergency stop on/off remotely.  This tool is also embedded in the Controller GUI, in the lower left hand corner.

By alt-clicking the button in ControllerGUI, a full sized window can be brought up and made any size you please.

No roboticist should be without a big red panic button! ;)

Error Messages

This very simple class allows code from the robot to bring messages up on the screen.  In the future, anything sent to serr (port 10002) will be brought up in a console similar to this to further streamline displaying important messages.

MATLAB Tools

Setting up TekkotsuMon - MATLAB

NOTE: The MATLAB tools are currently broken in version 1.5.  There is a work around: copy all of the *Listener.java files from tools/mon/org/tekkotsu/mon/ to tools/mon (with the MATLAB .m files)

In order to use TekkotsuMon, you need a reasonably fast computer running Matlab 6.5 or higher. If you want to modify or recompile the network communication classes (written in Java),  Change directory to

 tekkotsu_root/tools/mon

Edit config.m to specify the Aibo's IP address (conf.ip). Start Matlab, and at the prompt type

 run

It is important to launch MATLAB from within the tools/mon directory so that it can find related support files.  On Windows, you will need to make a shortcut to MATLAB and set the work path ("Start in:") to this directory (via right click -> Properties).

This should bring up a control window with buttons for various TekkotsuMon components found such as Raw Vision, Segmented Vision, Joint monitor, etc. Start the Aibo with Tekkotsu (wait for it to boot), and click on any button to start that component.


MATLAB Stuff - Currently Broken

Mecha Control

An earlier MATLAB prototype for the Java WalkGUI.  However, this interface has easily customizable buttons, which are currently set up to play sounds.  (To play sounds in the Java tools, use the Controller to open the File Access -> Play Sound menu, which will show a listing of all available sounds.)

Joint Graphs

These graphs show the position (blue) and force (red) on each of the joints.  This graph was taken during a run of the FollowHeadBehavior - look at the neck joints to see the different directions that the head was pointed in.

A similar graph is available for PID settings.

Vision Monitors

Similar to the Java tools, this should help you quickly get prototype MATLAB code for processing image data.

developer resources
Last modified: 2007-08-21