Welcome to ROSMASTER X3 repository

1. Module introduction and port binding usage

1.1. Introduction to voice control module

1.1.1, CSK4002 chip

The voice control module on ROSMASTER is developed based on the CSK4002 chip. CSK4002 is a high-performance, powerful computing power, low power consumption, and resource-rich AISoC developed and designed for the AloT field. It can be widely used in smart homes, smart appliances, and emerging consumer electronics industries.

• CSK4002 uses the Andes D1088 core. Its AI/DSP acceleration module MVA supports a variety of Neural Network operators and vector operations. It is deeply adapted to the iFlytek AI algorithm of HKUST and has a computing power of up to 128GOPS.

• Comes with 8M Flash, 8M PSRAM, 1M SRAM.

• Supports 8 channels of PDM audio input and 16 channels of I2S Audio Input data processing.

• Integrate rich mainstream peripheral interfaces: GPIO/UART/I2C/SPI/QSPI/SDIO/USB1.1/SDIO, etc.

• Equipped with low-latency embedded operating system Free RTOS, complete BSP driver, and comprehensive development tool resources.

1.1.2. Module features

• Far-field sound pickup: The front end adopts iFlytek's dual-microphone array algorithm, which can achieve 360-degree far-field 5m user sound pickup. Equipped with automatic gain for vocals, which can be adaptively adjusted according to the user's volume to ensure that the overall audio experience is consistent after noise reduction.

• Echo cancellation: During user interaction, when the device is broadcasting content or music, the user can wake up and interrupt the broadcast process for the next round of interaction, making the interaction experience more natural.

• Voice broadcast: Voice broadcast means that the user wakes up the device and speaks command words, and the device performs corresponding reply broadcast responses; or active prompts. The purpose of voice broadcast is to give feedback to the user by broadcasting a reply when the user issues a voice command or in an appropriate scenario.

• Offline command: When the device is awake, the user speaks a command word (instruction) within a specified range. After receiving the information, the voice module performs related processing based on the content of the command word, or transmits the content information to the host computer for related processing.

• Peripheral communication: The module receives input from the microphone for processing, and then communicates with other devices through USB, I2S, SDIO and other interfaces. There is also general programmable IO to communicate with other devices.

• Environmental noise reduction: It is widely used for environmental noise reduction in homes, cars, offices and other scenes. It can reduce noise while retaining human voice information to the greatest extent.

1.2. Using modules

1.2.1. Wiring

The module is connected to the ROSMASTER main control (or HUB board) through a universal Type-c data cable, and the speaker and module are connected through a PH2.0 data cable.

• As shown in the picture below, one end of the Micro-USB data cable is connected to the USB-connet port of the module, and the other end is connected to the port of the ROSMASTER motherboard (After this is plugged in, it cannot be modified after the later ports are bound. , please refer to the next section for details)

• As shown in the figure below, the PH2.0 data line port is connected to the Speaker port on the module.

  

1.2.2. Wake-up word

The wake-up word is "Hello, Xiaoya". When waking up, the speaking speed needs to be slowed down. If it is too fast, the module will not recognize it. After waking up the module, other command words can be recognized later. Within 20 seconds of waking up, there is no need to wake up again, just say the command word.

1.2.3. Command words

1. Voice control car movement

2. Voice control RGB light strip effect

3. Voice control color recognition

4. Voice control color tracking

5. Voice-controlled autonomous driving (line patrol)

6), Voice control multi-point navigation

1.3. Voice control module port binding

Preface: Because the ID device numbers of the ROS expansion board and the voice control module are the same, they cannot be modified according to the method in the previous tutorial.

Row ID device number binding. Not binding ports may cause port conflicts or device identification errors, and the bound ports cannot be changed at will

Change position, otherwise the binding will be invalid. This course is completed on the host machine. There is no need to bind it in the dock. This section uses the Raspberry Pi as the host machine to demonstrate. Before binding, you need to annotate the part involving myserial in the original host rule file. off, otherwise the voice board and the ROS expansion board will be recognized as the same device, and the terminal input will

x
sudo gedit /etc/udev/rules.d/serial.rules

As shown in the figure above, annotate the binding content related to myserial, save and exit, enter the following command in the terminal to reload the rules,

xxxxxxxxxx
sudo udevadm trigger
sudo service udev reload
sudo service udev restart

1.3.1. Bind the ROS expansion board device number

x
ll /dev/ttyUSB*
ll /dev/rplidar

• First, without connecting to the voice control board, you can get ttyUSB0 and ttyUSB1

  

  Here ttyUSB0 is the device number recognized by the radar, then ttyUSB1 is the device number of the ROS expansion board.

• Then, we first check the port information of the ROS expansion board, mainly to check the device path information and terminal input.

  x
  udevadm info --attribute-walk --name=/dev/ttyUSB1 |grep devpath
  

  The following information is obtained (the actual situation shall prevail here). The red box illustrates the path information of the device.

  

• Then, we modify the /etc/udev/rules.d/serial.rules file, first bind the port number of the ROS expansion board, and enter it in the terminal.

  x
  sudo gedit /etc/udev/rules.d/serial.rules 
  

  As shown below, replace the content

  Before replacement

  x
  KERNEL=="ttyUSB*",ATTRS{idVendor}=="1a86",ATTRS{idProduct}=="7523",MODE:="0777",SYMLINK+="myserial"
  

  After replacement

  x
  KERNEL=="ttyUSB*",ATTRS{devpath}=="1.4.3",ATTRS{idVendor}=="1a86",ATTRS{idProduct}=="7523",MODE:="0777",SYMLINK+=" myserial"
  

• Exit after saving, enter the following three commands in the terminal to reload the device,

  xxxxxxxxxx
  sudo udevadm trigger
  sudo service udev reload
  sudo service udev restart
  

1.3.2. Bind the voice board port number

• Connect to the voice board and enter the following command on the terminal to view the device number.

  x
  ll /dev/ttyUSB*
  

Here we find that the system recognizes the voice board as /dev/ttyUSB2, then we enter the following command to view the device path

information,

x
udevadm info --attribute-walk --name=/dev/ttyUSB2 |grep devpath

Get the following picture,

• Then, we modify the /etc/udev/rules.d/myspeech.rules file, bind the port number of the voice board, and enter it in the terminal,

  xxxxxxxxxx
  sudo gedit /etc/udev/rules.d/myspeech.rules
  

  Add content as shown in the picture below,

  x
  KERNEL=="ttyUSB*",ATTRS{devpath}=="1.4.2",ATTRS{idVendor}=="1a86",ATTRS{idProduct}=="7523",MODE:="0777",SYMLINK+=" myspeech"
  

• Exit after saving, enter the following statement in the terminal to reload the system device

  xxxxxxxxxx
  sudo udevadm trigger
  sudo service udev reload
  sudo service udev restart
  

1.3.3. Testing

Note where they are wired. This is fixed after binding. Do not change the USB interface position at will later, otherwise the system will not recognize the device, terminal input,

xxxxxxxxxx
ll /dev/myserial
ll /dev/myspeech

As shown in the figure above, if /dev/myserial and /dev/myspeech can be identified and the port numbers of the following ttyUSB* are inconsistent, it means the binding is successful.

Note: The bound ROS expansion board and voice board cannot be plugged into other ports, otherwise the device number will not be recognized. If both are connected to the HUB expansion board, the HUB board cannot be plugged into other ports. on the motherboard port.

1.3.4. Precautions

After binding the voice board, when entering the container, you need to mount /dev/myspeech to recognize the voice board in docker. For the mounting method, please refer to [07, Docker-05, Entering the Docker Container of the Robot - Editing Script]. content.