ROS2+智能导航小车 2024年7月25日 · 47 分钟
项目五：语音控制小车移动
在 ROS2 上集成科大讯飞中文语音识别，实现语音控制小车前进后退与转向
本文我们将在ROS2上集成科大讯飞的中文语音库，实现语音控制机器人小车前进后退，左转右转等。
一、linux系统（ubuntu）调用科大讯飞SDK实现语音识别
1. 科大讯飞官网
登录注册实名制
2. 点击控制台，创建应用
点击左侧的语音听写，右边下滑选择Linux，点击下载
选择Linux平台，普通版本，语音听写，SDK下载
此时将得到一个压缩包，选择的功能不同，文件名也不同，将这个压缩包放在ubuntu中
3. SDK包的处理
新建一个voice_ros2文件夹将压缩包中的文件放进去
进入sample目录的iat_online_record_sample目录下，运行下面的命令
source 64bit_make.sh
如果提示“alsa/asound.h:没有那个文件或目录”。解决办法：
sudo apt-get install libasound2-dev4. 检查环境
在测试之前，先检查自己的麦克风，以及虚拟机Linux的联网状态。
5.语音识别
在==bin==目录下执行对应的可执行文件了
./iat_online_record_sample
我写了一个调用的代码，可以不再键入0 1 ，而直接开始识别吗，并将识别结果保存在result.txt文件中
result.txt如下：
voice.py代码如下：
import subprocess
import multiprocessing
import time
def run_iat_online_record_sample(queue):
process = subprocess.Popen(["./bin/iat_online_record_sample"],
stdout=subprocess.PIPE,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
)
# Communicate with the process
stdout, _ = process.communicate(input=b"0\n1\n")
# Put the result into the queue
queue.put(stdout.decode('utf-8'))
def main():
while True:
# Create a queue for communication between processes
queue = multiprocessing.Queue()
# Start the process
process = multiprocessing.Process(target=run_iat_online_record_sample, args=(queue,))
process.start()
# Wait for the process to finish and get the result from the queue
process.join()
result = queue.get()
# Print the result
print("Result:", result)
# Save the result to a text file, clearing the file first
with open("result.txt", "w") as f:
f.write(result)
# Ask user whether to continue recognition
continue_recognition = input("是否继续识别？ (0: 结束, 1: 继续): ")
if continue_recognition == "0":
break
if __name__ == "__main__":
main()6. 问题
有一个可能出现的问题，
User
ubuntu22@ubuntu22-virtual-machine:~/ros2_car_ws/voice_ros2/bin$ ./iat_online_record_sample
./iat_online_record_sample: error while loading shared libraries: libmsc.so: cannot open shared object file: No such file or directory如果出现这个错误的话需要将下面这个路径的==libmsc.so==文件复制到==/usr/local/lib==路径下
执行下面的命令
sudo cp libmsc.so /usr/local/lib
sudo ldconfig
不出意外的话可以解决这个问题。
如果遇到其他错误代码请查看
二、使用麦克风实现语音控制
由于科大讯飞的语音包是C++代码编写，直接调用语音包的Demo，获得语音的识别结果，通过文件的形式将这个结果传递给ros2的键盘控制节点，以实现语音控制功能。
2.1 创建工作空间
我这边的工作空间命名为ros2_car_ws，并在这里创建个文件夹src，并将上一节的voice_ros2放到这里，如下图所示
src文件夹中的内容如下：
我们需要下载ROS2键盘控制功能包，在此基础上修改 teleop_twist_keyboard
2.2 编写代码
修改teleop_twist_keyboard.py文件
在键盘控制的代码前添加读取文件数据的代码
这里将刚刚识别到的语音过滤后存储在voice_command[0]中，以供后续使用，下面会通过判断voice_command[0]中的值来进行不同的操作
完整代码如下：
import sys
import threading
import time
import os
from std_msgs.msg import String
import geometry_msgs.msg
import rclpy
if sys.platform == 'win32':
import msvcrt
else:
import termios
import tty
msg = """
This node takes keypresses from the keyboard and publishes them
as Twist/TwistStamped messages. It works best with a US keyboard layout.
---------------------------
Moving around:
u i o
j k l
m , .
For Holonomic mode (strafing), hold down the shift key:
---------------------------
U I O
J K L
M < >
t : up (+z)
b : down (-z)
anything else : stop
q/z : increase/decrease max speeds by 10%
w/x : increase/decrease only linear speed by 10%
e/c : increase/decrease only angular speed by 10%
CTRL-C to quit
"""
moveBindings = {
'i': (1, 0, 0, 0),
'o': (1, 0, 0, -1),
'j': (0, 0, 0, 1),
'l': (0, 0, 0, -1),
'u': (1, 0, 0, 1),
',': (-1, 0, 0, 0),
'.': (-1, 0, 0, 1),
'm': (-1, 0, 0, -1),
'O': (1, -1, 0, 0),
'I': (1, 0, 0, 0),
'J': (0, 1, 0, 0),
'L': (0, -1, 0, 0),
'U': (1, 1, 0, 0),
'<': (-1, 0, 0, 0),
'>': (-1, -1, 0, 0),
'M': (-1, 1, 0, 0),
't': (0, 0, 1, 0),
'b': (0, 0, -1, 0),
}
speedBindings = {
'q': (1.1, 1.1),
'z': (.9, .9),
'w': (1.1, 1),
'x': (.9, 1),
'e': (1, 1.1),
'c': (1, .9),
}
def getKey(settings):
if sys.platform == 'win32':
# getwch() returns a string on Windows
key = msvcrt.getwch()
else:
tty.setraw(sys.stdin.fileno())
# sys.stdin.read() returns a string on Linux
key = sys.stdin.read(1)
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, settings)
return key
def saveTerminalSettings():
if sys.platform == 'win32':
return None
return termios.tcgetattr(sys.stdin)
def restoreTerminalSettings(old_settings):
if sys.platform == 'win32':
return
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_settings)
def vels(speed, turn):
return 'currently:\tspeed %s\tturn %s ' % (speed, turn)
def main():
settings = saveTerminalSettings()
rclpy.init()
node = rclpy.create_node('teleop_twist_keyboard')
# parameters
stamped = node.declare_parameter('stamped', False).value
frame_id = node.declare_parameter('frame_id', '').value
if not stamped and frame_id:
raise Exception("'frame_id' can only be set when 'stamped' is True")
if stamped:
TwistMsg = geometry_msgs.msg.TwistStamped
else:
TwistMsg = geometry_msgs.msg.Twist
pub = node.create_publisher(TwistMsg, 'cmd_vel', 10)
voice_command = [None] # Initializing as a list
spinner = threading.Thread(target=rclpy.spin, args=(node,))
spinner.start()
speed = 0.5
turn = 1.0
x = 0.0
y = 0.0
z = 0.0
th = 0.0
status = 0.0
twist_msg = TwistMsg()
if stamped:
twist = twist_msg.twist
twist_msg.header.stamp = node.get_clock().now().to_msg()
twist_msg.header.frame_id = frame_id
else:
twist = twist_msg
try:
print(msg)
print(vels(speed, turn))
while True:
print("当前工作路径:", os.getcwd())
with open('./voice_ros2/result.txt', 'r') as f:
for line in f:
if line.startswith('Result: ['):
start = line.find('[')
end = line.find(']')
if start != -1 and end != -1:
voice_command[0] = line[start + 1:end].strip()
print("voice_command", voice_command[0])
# Clearing the content of result.txt
open('./voice_ros2/result.txt', 'w').close()
break
key = getKey(settings)
# print("键盘控制按键输出", key)
if key in moveBindings.keys():
x = moveBindings[key][0]
y = moveBindings[key][1]
z = moveBindings[key][2]
th = moveBindings[key][3]
elif key in speedBindings.keys():
speed = speed * speedBindings[key][0]
turn = turn * speedBindings[key][1]
print(vels(speed, turn))
if (status == 14):
print(msg)
status = (status + 1) % 15
elif voice_command[0] is not None:
if voice_command[0] == "小车前进":
print("语音控制小车前进", voice_command[0])
x = moveBindings['i'][0]
y = moveBindings['i'][1]
z = moveBindings['i'][2]
th = moveBindings['i'][3]
elif voice_command[0] == "小车后退":
print("语音控制小车后退", voice_command[0])
x = moveBindings[','][0]
y = moveBindings[','][1]
z = moveBindings[','][2]
th = moveBindings[','][3]
elif voice_command[0] == "小车左转":
print("语音控制小车左转", voice_command[0])
x = moveBindings['j'][0]
y = moveBindings['j'][1]
z = moveBindings['j'][2]
th = moveBindings['j'][3]
elif voice_command[0] == "小车右转":
print("语音控制小车右转", voice_command[0])
x = moveBindings['l'][0]
y = moveBindings['l'][1]
z = moveBindings['l'][2]
th = moveBindings['l'][3]
elif voice_command[0] == "小车停":
print("语音控制小车停", voice_command[0])
x = moveBindings['k'][0]
y = moveBindings['k'][1]
z = moveBindings['k'][2]
th = moveBindings['k'][3]
voice_command[0] = None
else:
x = 0.0
y = 0.0
z = 0.0
th = 0.0
if (key == '\x03'):
break
if stamped:
twist_msg.header.stamp = node.get_clock().now().to_msg()
twist.linear.x = x * speed
twist.linear.y = y * speed
twist.linear.z = z * speed
twist.angular.x = 0.0
twist.angular.y = 0.0
twist.angular.z = th * turn
pub.publish(twist_msg)
# Print timestamp every second
time.sleep(1)
print("时间戳：", time.time())
except Exception as e:
print(e)
finally:
if stamped:
twist_msg.header.stamp = node.get_clock().now().to_msg()
twist.linear.x = 0.0
twist.linear.y = 0.0
twist.linear.z = 0.0
twist.angular.x = 0.0
twist.angular.y = 0.0
twist.angular.z = 0.0
pub.publish(twist_msg)
rclpy.shutdown()
spinner.join()
restoreTerminalSettings(settings)
if __name__ == '__main__':
main()
2.3 编译运行
我们一共需要运行2个终端
- 第一个终端，运行键盘控制节点
cd ros2_car_ws
colcon build --packages-select teleop_twist_keyboard
source install/setup.bash
ros2 run teleop_twist_keyboard teleop_twist_keyboard编译时会出现下面的问题，这是因为存在和原本的控制包覆盖的问题，不需要管，我们只在这个ROS2的工作空间中使用
lll@lll-virtual-machine:~/ros2_car_ws$ colcon build --packages-select teleop_twist_keyboard
[0.131s] WARNING:colcon.colcon_core.package_selection:Some selected packages are already built in one or more underlay workspaces:
'teleop_twist_keyboard' is in: /opt/ros/humble
If a package in a merged underlay workspace is overridden and it installs headers, then all packages in the overlay must sort their include directories by workspace order. Failure to do so may result in build failures or undefined behavior at run time.
If the overridden package is used by another package in any underlay, then the overriding package in the overlay must be API and ABI compatible or undefined behavior at run time may occur.
If you understand the risks and want to override a package anyways, add the following to the command line:
--allow-overriding teleop_twist_keyboard
This may be promoted to an error in a future release of colcon-override-check.
Starting >>> teleop_twist_keyboard
Finished <<< teleop_twist_keyboard [0.69s]
Summary: 1 package finished [0.78s]
- 第二个终端，运行voice.py文进行语音识别
在voice_ros2文件夹中运行
python3 voice.py
然后就可以通过语音控制小车移动了
在右侧终端按1进行语音识别，此时将识别到小车前进的命令并打印，在左侧终端按回车健获取result中的命令，将输出voice_command 小车前进，此时再按键回车，将输出语音控制小车前进 小车前进并且小车开始移动。
三、启动小车
ssh远程登录到树莓派后的终端
ssh user@192.168.1.100
# 然后输入密码即可登录随后使用tmux分割终端，一共需要3个终端
# 切换窗口
ctrl+b c: 创建一个新窗口（状态栏会显示多个窗口的信息）
ctrl+b p: 切换到上一个窗口（按照状态栏的顺序）
ctrl+b n: 切换到下一个窗口
ctrl+b w: 从列表中选择窗口（这个最好用）3.1 第一个终端：
小车控制
cd ros2_car_ws
source install/setup.bash
ros2 run micro_ros_agent micro_ros_agent udp4 --port 8888 -v63.2 第二个终端：
cd ros2_car_ws
colcon build --packages-select teleop_twist_keyboard
source install/setup.bash
ros2 run teleop_twist_keyboard teleop_twist_keyboard3.3 第三个终端：
在voice_ros2文件夹中运行
python3 voice.py