GeeKee CeeBee

Welcome to GeeKee CeeBee's Page: House of Mechatronics Projects & Lessons.
Contact Email: Ceebee1108@gmail.com
Follow me on Youtube
__________________________________________________________________________________________________________________________________

Lane Keeping Assist System Using Steering Wheel

and CARLA Simulator

__________________________________________________________________________________________________________________________________

Typical Lane Keeping Assist System

Lane-keeping assist system is responsible for detecting the lane departure and correcting the vehicle trajectory by adjusting the steering wheel by small angle in semi-autonomous vehicle. General elements and sub-modules of the LKAS are shown in Figure 1. Camera detects lane markings on the lane, State Observer calculates the vehicle dynamics which is used to calculate steering wheel angle and torque required. Further human-machine role negotiation algorithm is implemented to control the steering wheel of the vehicle. Apart from hardware issues, the challenge is to find the “correct code of communication” between human and machine [3].

__________________________________________________________________________________________________________________________________

CARLA Driving Simulator

CARLA simulator is an open-source driving simulator developed using C++ language and Unreal Game Engine vehicle physics. CARLA supports development, training and validation of autonomous urban driving systems that provides 3D photorealistic simulation as shown in Figure 5 [8]. CARLA allows a server-client interface that can be accessed using Python API as shown in Figure 6. CARLA world object allows easy access to built-in town maps with lane markings, road curbs, traffic lights, etc. and actors like vehicles and pedestrian. In addition, kinematic sensor for steering, braking and velocity measurements are also available including camera sensor for computer vision, plus the GPS location sensor [8]. In order to automatically extract markings in CARLA simulator, semantic segmentation camera sensor can be used. Lane lines can be highlighted using the detect_lanes function in CARLA as shown in Figure 7 [10].

__________________________________________________________________________________________________________________________________

MATLAB - Vehicle Lateral and Longitudinal Controller

Using kinematic vehicle dynamic equations and longitudinal vehicle forces equations, plant models are constructed in below Simulink model shown in Figure 10.

__________________________________________________________________________________________________________________________________

Steering Wheel Construction and CARLA Communication

In order to construct an ergonomic steering wheel simulator, an existing product, called Interact V3 Racing Wheel Controller, was repurposed. All the plastic housing and parts were salvaged from an old off-the-shelf gaming steering wheel controller. This plastic housing was fitted with rotary potentiometer for position sensing with the help of pair of gears, high rpm DC motor, and motor driver L298n and the planetary gear reduction (14:1) attachment to form a desired steering wheel simulator shown in Figure 11. Initial decision to use rotary quadrature encoder was scrapped due its compromised position accuracy. All components were mechanically attached to the plastic steering wheel as shown in Figure 12. Three shaft couplers were used to connect various components to each other.

In preparation to facilitate proper communication between steering wheel-pedals assembly and CARLA, the wiring and flow diagram shown in Figure 13 was developed and implemented. CARLA 0.95 Windows version and Python 3.7 were used in this study. Pedals assembly and Steering wheel assembly were connected to their individual Arduino microcontrollers. StandardFirmata code was uploaded to the Arduino interacting with pedals assembly. StandardFirmata is available in Arduino IDE under File > Examples > Firmata > StandardFirmata. Python PyFirmata [15] library was imported inside the main Python API code to form communication between pedals hardware and CARLA.

An example code file called manual_control.py available with CARLA [18] was modified to develop the main Python API code. Modifications include the means for receiving analog inputs from pedals hardware and interrupt signals from Steering wheel encoder. This API is responsible to communicate with the CARLA simulator world which consists of the all objects, blueprints, maps and actors. In other words, this API is simply an interface which defines communication between Steering wheel-pedals hardware and CARLA simulator. This API also contains the control algorithms used to implement vehicle longitudinal and lateral controller and also human machine controller.

Connect the Arduinos to the host PC via USB and determine their COM ports respectively. Assign COM ports in the main API code, respectively. To run the simulation, first run the CARLA server using “CarlaUE4.exe” file located in the CARLA folder. Note that Carla server will keep running in the background. To run the API open Command Prompt and jump to the folder directory where the API files is stored. And then type “py -3.7 –m API_file_name.py” and press Enter. A new window will open where the hardware get to interact in real time with CARLA world.

__________________________________________________________________________________________________________________________________

Controllers

Overall control algorithm layout is shown in Figure 14. Furthermore, waypoints depiction shown in Figure 15 were used to determine center of the lane at each time step of the simulation. Using the dot product and cross product of the car velocity vector and vector between the location of the car and location of the waypoint, the angle was determined. The goal of the lateral controller is to reduce this angle to zero. Longitudinal controller was to control the throttle and braking parameter of the vehicle as to maintain the set velocity of 11m/s. Since the potentiometer analog range was mapped to the 300 degrees rotational range. PD controller was used to implement tracking controller for the Steering wheel to track vehicle’s steering angle from the simulation.

__________________________________________________________________________________________________________________________________

RESULTS- Vehicle Longitudinal and Lateral Controller

Longitudinal controller results are shown in Figure 19 and 20. Actual drive cycle data (see Appendix B) was used for simulation of 150s. Controller tracks the drive cycle velocity very well. Moreover, the changes caused in the throttle/ braking force of the vehicle due to the changes in its velocity seem realistic. As the velocity of the vehicle decreases the propulsion forces also decreases, and vice versa. In addition, these results align with results presented by Kumbhani et al. [7].

Moreover, lateral controller results are shown in Figure 21 and 22. To demonstrate the controller’s effectiveness during vehicle’s lateral position tracking, multistep velocity input was used in the simulation of 60s. During lane change small changes in vehicle’s lateral position is expected. Thus, the controller is tested at 5m change in lateral position while driving at various speed. Controller behaves quite well while tracking the lateral position. Plus, changes in vehicle’s steering angle validates vehicle realistic behavior. Once again, these results align with the results presented by Kumbhani et al. [7].

__________________________________________________________________________________________________________________________________

RESULTS- Steering Wheel and Pedals Testing with CARLA

Communication of steering wheel and pedals hardware with CARLA simulator was successfully achieved. Complete steering wheel setup while in use is shown in Figure 23. Turning the steering wheel caused the vehicle to turn as intended in the simulation. Throttle and brake pedals also performed as intended. The content of the example code allowed an overlay over the simulation screen which displays status bars for steering angle, acceleration and braking measurements. Synchronize mode was enabled to perform event synchronization between python API and the server simulation. Overall open loop control worked as intended.

Control algorithms implementation was also tested. Test setup is shown in Figure 24. Results can be seen in Figures 25-28. Longitudinal controller was used to maintain the vehicle velocity at 11m/s. Lateral controller was used to keep the vehicle within the lane, more precisely on the center of the lane. Human input was added to rotate the steering wheel and with the help of switching algorithm vehicle control was handed over to the operator while automatic controller renders ineffective. Overall, control algorithm successfully recognized the human operator as superior controller by determining the amount of counter torque applied to the steering wheel by the human operator.

__________________________________________________________________________________________________________________________________

__________________________________________________________________________________________________________________________________

__________________________________________________________________________________________________________________________________