smach introspection server fails in electric

asked 2011-08-30 20:34:37 -0600

sebastian
76 ●1 ●3 ●6

Hi everyone,

I am using smach in electric with the introspection server and the smach_viewer.py.

In electric the smach_viewer.py does not work for me (it just shows the machine but does not update in which state the machine is) and the script running the introspection server periodically shows the following exception:

Exception in thread server_name:status_publisher:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/opt/ros/electric/stacks/executive_smach/smach_ros/src/smach_ros/introspection.py", line 160, in _status_pub_loop
    self._publish_status('HEARTBEAT')
  File "/opt/ros/electric/stacks/executive_smach/smach_ros/src/smach_ros/introspection.py", line 225, in _publish_status
    self._status_pub.publish(state_msg)
  File "/opt/ros/electric/stacks/ros_comm/clients/rospy/src/rospy/topics.py", line 695, in publish
    self.impl.publish(data)
  File "/opt/ros/electric/stacks/ros_comm/clients/rospy/src/rospy/topics.py", line 872, in publish
    serialize_message(b, self.seq, message)
  File "/opt/ros/electric/stacks/ros_comm/clients/rospy/src/rospy/msg.py", line 151, in serialize_message
    msg.serialize(b)
  File "/opt/ros/electric/stacks/executive_smach/smach_msgs/src/smach_msgs/msg/_SmachContainerStatus.py", line 120, in serialize
    buff.write(struct.pack('<I%ss'%length, length, _x.encode()))
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)

Without the introspection sever or when running smach from diamondback the exception does not show up and everything works as expected. Does anyone have a clue what the problem is?

I used the script "examples/user_data2.py" from the smach_tutorials package and added the introspection server as follows:

--- user_data2.py   2011-08-31 09:41:46.108424577 +0200
+++ user_data_sis.py    2011-08-31 10:13:34.652678420 +0200
@@ -59,7 +59,11 @@


     # Execute SMACH plan
+   sis = smach_ros.IntrospectionServer('server_name', sm, '/SM_ROOT')
+   sis.start()
     outcome = sm.execute()
+    rospy.spin()
+    sis.stop()


 if __name__ == '__main__':

Thanks for your help!

edit retag flag offensive close merge delete

Comments

This looks like somewhere there's a unicode issues. Is there any unicode used in your smach state machine states? 0x80 is not a valid ascii code as the backtrace suggests.

tfoote ( 2011-09-29 05:25:36 -0600 )edit

add a comment

4

answered 2011-10-28 05:56:07 -0600

kwc
12244 ●54 ●90 ●132 http://kwc.org/

I suspect this is a regression caused by a Python 3k compatibility patch in the message generators. I have backed this out in r15301, but it will take awhile to get this fix deployed as it will required a full deb rebuild.

edit flag offensive delete link

Comments

If so, it looks to me like the Python 3k compatibility is just more restrictive. Really the pickled user data shouldn't be "string" it should be some byte array.

jbohren ( 2011-10-28 06:01:55 -0600 )edit

realistically, for true py3k compatibility, the message generator needs to be rewritten to start using "bytes" and use the distinctions that the Python 2.6+ APIs now provide. This is a significant undertaking, so for now, backing out the Py3k patch.

kwc ( 2011-10-28 06:07:22 -0600 )edit

add a comment

2

answered 2011-11-08 22:14:11 -0600

michikarg
2108 ●19 ●37 ●52 http://hcai.in.tum.de/...

Hi,

i just rechecked the problem with the ascii-encoding and it´s actually less a problem than i thought it is. It was actually bug in my patch. On

http://docs.python.org/dev/whatsnew/3.2.html#porting-to-python-3-2, it says:

"struct.pack() now only allows bytes for the s string pack code. Formerly, it would accept text arguments and implicitly encode them to bytes using UTF-8. This was problematic because it made assumptions about the correct encoding and because a variable-length encoding can fail when writing to fixed length segment of a structure."

So since Python3.2 struct.pack did an implicit encoding of string-data using UTF8-encoding which since Python3.2 has to be done explicitly. In my patch i therefore added "s.encode()" which uses the python standard string encoding (which apparently was set to "ascii" at Manuels computer) instead of "utf-8" like it did before. So changing this to s.encode("utf-8") should produce the same behaviour in Python2.6 and also work with Python3.2 even if your standard string-encoding is set to "ascii".

Sorry about that.

@Ken: Do you want me to send a patch for this issue?

Cheers

Michi

edit flag offensive delete link

Comments

line 643 in genpy.py should look like this: yield pack2("'<I%ss'%length", "length, %s.encode("utf-8")"%var)

michikarg ( 2011-11-08 22:16:00 -0600 )edit

str.encode() is stateful and can't be repeated for strings previously encoded -- following up in e-mail to discuss more.

kwc ( 2011-11-09 04:23:31 -0600 )edit

add a comment

1

answered 2011-10-28 05:57:51 -0600

jbohren

5799 ●55 ●87 ●102 http://dscl.lcsr.jhu.e...

updated 2011-10-28 09:55:58 -0600

I posted this to ros-users as well, but I'll add it here for completeness.

It appears that in Electric, rospy enforces that strings in ROS messages be ASCII strings, so the output of the python pickle module can't be encoded. To fix this we should be able to just change the "local_data" field in https://kforge.ros.org/smach/executive_smach/file/d79964e4c22f/smach_msgs/msg/SmachContainerStatus.msg">SmachContainerStatus from:

12 # A pickled user data structure
13 string local_data

to:

12 # A pickled user data structure
13 uint8[] local_data

I don't currently have the time to test this fix, but I'm pretty sure this is the problem.

edit flag offensive delete link

Comments

actually, should be uint8[]. "byte" is deprecated and shouldn't be used (it's been hard to actually remove due to widespread use and md5sum compatibility)

kwc ( 2011-10-28 06:08:20 -0600 )edit

fixed in edit

jbohren ( 2011-10-28 09:56:21 -0600 )edit

add a comment

1

answered 2011-11-06 05:55:25 -0600

michikarg
2108 ●19 ●37 ●52 http://hcai.in.tum.de/...

Hey guys,

sorry for my late reply. I guess this is indeed caused by trying to introduce partial Python3-compatibility into ROS. The "encode()" and "decode()" parts had to be added because of the new string representation in Python 3 (Unicode strings and bytes for data instead of unicode and 8-bit strings, see http://docs.python.org/release/3.0.1/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit).

However it seems like the _x.encode() uses the current standard string encoding to get from string to bytes which seems to be 'ascii' in your case while on my computer it is 'utf-8'. This could be the cause why Wim could not reproduce the problem. To test this, could you set your default string encoding to utf-8 and check if you still get the error?

You can set your default encoding using:

sys.setdefaultencoding('utf-8')

I am not familiar with the smach viewer, but i would be interested what kind of data you are sending SmachContainerStatus-messages...

If this works, there is the possibility that we force the encoding to be utf-8 in the messages... of course, we would have to set the decode-functions to utf-8 too....

Hope this helps and sorry again for the late reply...

Cheers

Michi

edit flag offensive delete link

add a comment

0

answered 2011-09-29 01:33:46 -0600

manabu
46 ●1 ●3 ●4

I'm also in this trouble. This occurs when run the patched user_data2.py and subscribing "/server_name/smach/container_status" topic.

File "/opt/ros/electric/stacks/executive_smach/smach_msgs/src/smach_msgs/msg/_SmachContainerStatus.py", line 120, in serialize
    buff.write(struct.pack('<I%ss'%length, length, _x.encode()))
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)

When I modified the introspection.py, the error does not occur.

diff -r 46cddfcd1bf3 smach_ros/src/smach_ros/introspection.py
--- a/smach_ros/src/smach_ros/introspection.py  Tue Aug 02 10:31:19 2011 -0700
+++ b/smach_ros/src/smach_ros/introspection.py  Thu Sep 29 22:31:48 2011 +0900
@@ -219,7 +219,7 @@
                     path,
                     self._container.get_initial_states(),
                     self._container.get_active_states(),
-                    pickle.dumps(self._container.userdata._data,2),
+                    "",
                     info_str)
             # Publish message
             self._status_pub.publish(state_msg)

edit flag offensive delete link

Comments

This is throwing the baby out with the bathwater. It's removing the part of the status message that includes the user data.

jbohren ( 2011-10-28 05:50:00 -0600 )edit

add a comment

0

answered 2011-10-25 20:01:17 -0600

marodga
1

Hi all, I'm having the same trouble that Sebastian... Has anyone solved it?

edit flag offensive delete link

add a comment

0

answered 2011-09-12 04:52:37 -0600

Wim
2915 ●14 ●34 ●43

I haven't been able to reproduce this problem on the latest Electric Debians. Are you sure you're not mixing Electric and Diamondback? It's possible that the Electric viewer cannot connect to a Diamondback state machine (or vice versa).

edit flag offensive delete link

Comments

I am not mixing electric and diamondback, my debs are up to date (ros-electric-executive-smach: 1.0.2-s1313705059~natty, ros-electric-executive-smach-visualization 1.0.1-s1315010991~natty). I tried it on 64 and 32bit machines -> The Exception does still occur (a colleague of mine has the same issue)

sebastian ( 2011-09-12 22:55:18 -0600 )edit

add a comment

smach introspection server fails in electric

Comments

7 Answers

Comments

Comments

Comments

Comments

Comments

Question Tools

Stats

Related questions

smach introspection server fails in electric edit

Comments

7 Answers

Comments

Comments

Comments

Comments

Comments

Question Tools

Stats

Related questions

smach introspection server fails in electric