Roboy Pinky: Avatar Mode 101

Devices overview

Name

Nmap output

sudo nmap -nsP 10.180.1.0/24

Login credentials

Function

Uses ROS

Name

Nmap output

sudo nmap -nsP 10.180.1.0/24

Login credentials

Function

Uses ROS

Calculon-9

 

IP of the device is always listed above its name

this is the output on LAN using the white Tp-Link Ethernet adapter

luis

 

  • ROS master

  • kindyn (tendon control)

  • pybullet (joint control)

  • ball-and-socket joint position estimator

NUC

(replacement for Calculon-9)

 

 

roboy

 

  • ROS master

  • kindyn (tendon control)

  • pybullet (joint control)

  • ball-and-socket joint position estimator

FPGA

root

 

  • motor control

  • sensor data acquisition

  • runs roboy_plexus ROS node

Jetson Xavier

roboy

 

  • Node-Red

  • animus_server

  • tcp serial node (ROS wheelchair connection)

  • websocket server node (ROS face connection)

Jetson Nano

roboy

 

  • animus cloud replica

Running the system

  1. Inspect hardware connections integrity

    1. button box switches are all in OFF position

      1. OFF = towards body part name label

      2. OFF = O for 5V and 12V

    2. fuse on the left side of the wheelchair is green (OFF)

    3. FPGA power WAGO is connected (bottom right on the Core Board)

    4. micromatch connectors on the FPGA’s Core Board are properly plugged in

    5. micromatch connectors on the motorboards (front and back side) are properly plugged in

  2. Power the system

    1. connect the black socket plug to AC →

      1. green light appears on the DC-DC convertor below the wheelchair

      2. Ethernet switch lights turn on

      3. Core Board fan starts spinning and FPGA’s blue light turns on

      4. in the middle of the chest the white microphone’s tiny blue light turns on

    2. turn on the beamer

      1. push the switch to ON position

      2. press and hold power button for a few seconds

         

    3. turn on the speaker by turning a knob on a board located in the upper right side on the back of the robot

  3. Discover devices on the network and note their IPs

    1. run the IP finder script on Calculon-9 (currently mapping for 10.180.1.0, edit the script if needed)

      ~/ip_finder.sh


      OR

    2. run the IP finder on another machine


      OR

       

    3. use sudo nmap -nsP 10.180.1.0/24 and see “Nmap output” from the Devices Overview table

  4. Update ROS environmental variables for devices marked as “Uses ROS” in the Devices overview table

    1. open ~/.bashrc with an editor

    2. update ROS_MASTER_URI line

    3. update ROS_IP line (if necessary).
      If the ROS_IP is wrong, no ROS messages from this device will be recognized

    4. source ~/.bashrc

  5. Update animus cloud replica IP

    1. note the IP of the Jetson Nano board

    2. update cloud_ip on the Jetson Xavier

    3. update cloud_ip on the Operator’s PC in C:/Users/Roboy/.cyberselves/animus/cloud_ip file

  6. Autostart ROS master ( on Calculon-9 / NUC)

     

  7. Run rviz on a PC with a screen (ideally on LAN too), manually move the robot arms and head and confirm that the visualization in RViz is matching the physical robot state.

  8. Autostart on Jetson Xavier

    Confirm that animus_server is working properly by scrolling to the start of the command output (press Ctrl+A ü ⬆ ) and look for the following lines:

    2021/08/18 17:32:11 Registration reply: 3 action from client 1: at 1629300731016952374
    2021/08/18 17:32:11 Robot registered successfully with cloud platform.

  9. Turn on wheelchair by flipping the fuse on the left side of the wheelchair from green to red →

    1. ESP on the very bottom of the wheelchair will have the blue LED on

    2. “Subscribing to .. “ logs appear in the wheelchair TCP node tmux window on Jetson Xavier

  10. Restart face

  11. Turn on the body parts, 5V, 12V

  12. Run Motor Check from the Node-red GUI on http://<jetson xavier IP>:1880/ui (e.g. http://192.168.1.104:1880/ui) for

    1. shoulder left

    2. shoulder right

    3. head

  13. Initialize the robot with Node-Red

  14. Roboy Avatar is ready for telepresence!

Troubleshooting

Animus modality “Not Open”

Network is

Network should

Network is

Network should

LAN cable roll (avatar) → Teltonica Router

LAN cable roll (avatar)→ XPRIZE Avatar Network

Jetson Nano → Teltonica Router

Jetson Nano → XPRIZE Avatar Network

Operator PC (NUC) → XPRIZE Operator Network

Operator PC (NUC) → XPRIZE Operator Network

Symptoms visible in the animus_server logs:
Modality - not open - ...

Modality - not open - ...

Caused (likely) by animus client’s (on the NUC) inability to retrieve data from the animus server (Jetson Xavier).

Fix routine

  1. Connect the LAN cable roll to XPRIZE Avatar Network

  2. Connect Jetson Nano to XPRIZE Avatar Network

  3. run ./ip_finder.sh on Calculon-9

  4. update ROS_MASTER_URI for devices on ROS (see top table) to Calculon-9’s IP

  5. update wheelchair tcp socket IP to Jetson Xavier’s IP

  6. update face websocket IP to Jetson Xavier’s IP

FPGA debugging

In case the FPGA doesn’t show up on the network connect it to a laptop with a USB-mini cable on the side of the LAN (top when mounted on the robot) and run:

Then power cycle the FPGA.

You should see the booting sequence in the terminal running screen command.

Check if it’s getting an IP by ifconfig edit /etc/network/interfaces if necessary.

In case there’s no output on the screen , power cycle it again. With no output after few power cycles exchange the SD card and/or the FPGA board.

Manual joint control

Option 1

One can send joint targets to /roboy/pinky/simulation/joint_targets ROS topic. This will result in smooth and slow motions, since pybullet will take care of the interpolation between the target you sent and the current joint state. For example, to test a single joint:

Option 2

Another option is to send joint commands /roboy/pinky/control/joint_targets. This might however result in unexpectedly abrupt & fast motions. Not recommended. Use at your own risk

Motorboard exchange

See Config Files | Updating motor ids

For instructions to flash a new ID for motorboards refer to https://devanthro.atlassian.net/wiki/spaces/R3/pages/2049474573

Other

See https://devanthro.atlassian.net/wiki/spaces/~865154073/pages/2470608897