Difference between revisions of "OHMC2019 Software instructions"

From Open Hardware Miniconf
Jump to: navigation, search
(→‎Software installation: Raspberry Pi)
(→‎Neopixels)
 
(27 intermediate revisions by 2 users not shown)
Line 6: Line 6:
  
 
== Software installation: Laptop / Desktop ==
 
== Software installation: Laptop / Desktop ==
 +
 +
[http://docs.donkeycar.com/guide/install_software Extensive DonkeyCar documentation]
  
 
* apt-get install -y virtualenv  # Note: You may be using a different software installer
 
* apt-get install -y virtualenv  # Note: You may be using a different software installer
 
* mkvirtualenv donkeycar -p python3
 
* mkvirtualenv donkeycar -p python3
 +
* pip install tensorflow==1.8.0  # Note: Probably requires Python 3.5 or 3.6.  People are having problems with Python 3.7
 +
(if you get errors, you can try (re-) installing pip: python -m pip install --upgrade pip )
 +
 +
On debian, if virtualenv isn't there, try this instead
 +
* virtualenv donkeycar -p python3
 +
* cd donkeycar
 +
* export PATH=`pwd`/bin:$PATH
 
* pip install tensorflow==1.8.0
 
* pip install tensorflow==1.8.0
  
−
* cd $HOME
+
Then proceed:
 +
 
 
* git clone https://github.com/autorope/donkeycar
 
* git clone https://github.com/autorope/donkeycar
 
* pip install -e ./donkeycar
 
* pip install -e ./donkeycar
 +
 +
Note: You can manually install Python 3.6 as follows, but then you can't use mkvirtualenv ...
 +
 +
* yum install python36
 +
* python3.6 -m venv ~/virtualenvs/donkeycar
 +
* source ~/virtualenvs/donkeycar/bin/activate
  
 
== Software installation: Raspberry Pi ==
 
== Software installation: Raspberry Pi ==
  
−
Your DonkeyCar (Raspberry Pi) is already pre-installed.  This section is for reference only.
+
'''Your DonkeyCar (Raspberry Pi) is already pre-installed.  This section is for reference only.
 +
'''
 +
 
 +
[http://docs.donkeycar.com/guide/install_software/#get-the-raspberry-pi-working Extensive DonkeyCar documentation]
  
 
* sysctl -w net.ipv6.conf.all.disable_ipv6=1    # Reconnect via ssh afterwards
 
* sysctl -w net.ipv6.conf.all.disable_ipv6=1    # Reconnect via ssh afterwards
Line 44: Line 63:
 
* pip install adafruit-pca9685
 
* pip install adafruit-pca9685
 
* pip install picamera
 
* pip install picamera
 +
 +
== Finding your car on the network ==
 +
 +
Your DonkeyCar (Raspberry Pi) has been pre-configured to connect to the LCA2019 network.  But, which IP address ?
 +
 +
Every car has a unique hostname, from ''ohmc_01'' to ''ohmc_32''.  The micro-SD card adapter is labeled with your car name.
 +
 +
However, that still isn't enough to find your car.
 +
 +
So, there is a ''cron'' job that every minute sends an MQTT message to ''test.mosquitto.org'' with your car's name, IP address and a timestamp.
 +
 +
Install an MQTT client, as follows ...
 +
 +
* apt-get install mosquitto-clients
 +
 +
Use the following command to read the MQTT messages ...
 +
 +
* mosquitto_sub -h test.mosquitto.org -t 'ohmc/#' -v
 +
 +
  ohmc/ohmc_01 10.193.2.69 Mon 11 Jun 07:08:01 UTC 2018
  
 
== Driving the car manually ==
 
== Driving the car manually ==
 +
 +
'''Caution: Put your car "on blocks" (wheels off the ground) the first time you try driving it'''
 +
 +
[http://docs.donkeycar.com/guide/get_driving Extensive DonkeyCar documentation]
 +
 +
IP_ADDRESS=xxx.xxx.xxx.xxx  # Found as above
 +
 +
* ssh pi@$IP_ADDRESS  # Note: Raspberry Pi default username: ''pi'' and password: ''raspberry''
 +
* cd ohmc_car
 +
* python manage.py drive
 +
 +
  loading config file: /home/pi/play/roba_car/config.py
 +
  config loaded
 +
  PiCamera loaded.. .warming camera
 +
  Starting Donkey Server...
 +
  You can now go to http://xxx.xxx.xxx.xxx:8887 to drive your car.
 +
 +
With a desktop web browser, the user interface provides a virtual joystick (right-hand frame) that you can use to drive the car ... altering the steering and throttle values.
 +
 +
The mobile web browser, the user interface allows you to drive by tilting the phone left-right for steering and forwards-backwards for throttle.  For safety, you must press the [Start Vehicle] / [Stop Vehicle] toggle button to enable control.
 +
 +
== rPi reboots and Auto-starting donkeycar without logging in ==
 +
 +
If your rPi reboots when you're driving, check the following things:
 +
# did you swap the servo wires between steering and power. Turn the ESC off, unplug one servo wire, turn the ESC back on (little toggle switch), and test steering. Make sure that steering does not activate the wheel motors. 
 +
# is your battery low? are you accelerating too quickly? 
 +
Because the battery has a high internal resistance, if you demand a lot of power too quickly, the voltage on the battery drops enough that the rPi gets below its critical voltage (when you add the voltage drop from the voltage converter), and the rPi reboots.
 +
 +
To restart donkey server automatically, do the following: 
 +
* Add this to /etc/rc.local: 
 +
su - pi -c 'cd ~/ohmc_car; python manage.py drive &>/tmp/out' & 
 +
* move 'source ~/env/bin/activate' from ~/.bashrc to ~/.profile
 +
 +
== Steering and throttle calibration ==
 +
 +
To save time at the workshop, you won't need to calibrate your car's steering and/or throttle.
 +
 +
However, you may get better results and can perform calibration when you have time ... [http://docs.donkeycar.com/guide/calibrate by following the DonkeyCar instructions]
  
 
== Data acquisition for the Neural Network ==
 
== Data acquisition for the Neural Network ==
 +
 +
Once you are driving your car confidently around a track ... it is time to acquire training data for the Neural Network.  DonkeyCar operates at 10 frames per second, capturing a 160x120 image, along with steering angle and throttle value.  This is all stored in the ''$HOME/ohmc_car/tub/'' directory.
 +
 +
Before training, it is a good idea to clean out previous data.  Don't just remove all the files in the ''tub/'' directory !  The ''tub/meta.json'' file is important.
 +
 +
[http://docs.donkeycar.com/guide/train_autopilot Extensive DonkeyCar documentation]
 +
 +
Perform the same commands as for manual driving ...
 +
 +
* ssh pi@$IP_ADDRESS
 +
* cd ohmc_car
 +
* python manage.py drive
 +
 +
Then via the web browser press the [Start Recording] button ... drive the car around a track ... then press the [Stop Recording] button.
 +
 +
It is recommended that you collect between 5K and 20K frames.  At 10 frames per second, that is between 500 and 2,000 seconds of driving.  Make sure that you drive clockwise and anti-clockwise.
 +
 +
You will need to type this command just once on your DonkeyCar (Raspberry Pi)to provide a directory on your laptop for your training data ...
 +
 +
* rsync -av pi@<car_ip>:ohmc_car .
 +
 +
When finished acquisition, then transfer the data from the DonkeyCar (Raspberry Pi) to your laptop / desktop for training the Neural Network.
 +
 +
* rsync -av pi@<car_ip>:ohmc_car/tub ohmc_car/tub_$DATE
  
 
== Training the Neural Network ==
 
== Training the Neural Network ==
  
−
== Self-driving the car ==
+
Once training data has been copied to your laptop / desktop, you can begin training the Neural Network.
 +
 
 +
[http://docs.donkeycar.com/guide/train_autopilot Extensive DonkeyCar documentation]
 +
 
 +
Run these commands on your laptop / desktop to train the Neural Network ...
 +
 
 +
* workon donkeycar  # For those who have set-up a virtualenv
 +
* cd ohmc_car
 +
* python manage.py train --tub $HOME/ohmc_car/tub_$DATE --model ./models/model_$DATE.hdf5
 +
 
 +
  using donkey version: 2.5.7 ...
 +
  loading config file: /Users/andyg/play/ai/roba_car/config.py
 +
  config loaded
 +
  tub_names ./tub_2019-01-15c
 +
  train: 5740, validation: 1436
 +
  steps_per_epoch 44
 +
  Epoch 1/100
 +
  2019-01-21 13:08:49.507048: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
 +
  43/44 [============================>.] - ETA: 0s - loss: 58.5130 - angle_out_loss: 30.3421 - throttle_out_loss: 86.6839     
 +
  Epoch 00001: val_loss improved from inf to 0.19699, saving model to ./models/roba0_2019-01-16c.hdf5
 +
  44/44 [==============================] - 38s 874ms/step - loss: 57.1887 - angle_out_loss: 29.6601 - throttle_out_loss: 84.7172 - val_loss: 0.1970 - val_angle_out_loss: 0.3230 - val_throttle_out_loss: 0.0710
 +
 
 +
On a modern laptop, each epoch will take around 30 seconds to complete.  For up-to 100 epochs.  Typically, you can expect around 20 to 40 epochs before the Neural Network stop learning.  That is around 10 to 20 minutes of training time.
 +
 
 +
The training command creates the Neural Network weights that represent what your DonkeyCar has "learned".
 +
 
 +
When training has completed, copy the trained model back to your DonkeyCar (Raspberry Pi)
 +
 
 +
* scp $USERNAME@$HOSTNAME:ohmc_car/models/model_$DATE.hdf5 models
 +
 
 +
== Self-driving ... the ultimate goal ==
 +
 
 +
Once your trained model has been copied back onto the DonkeyCar (Raspberry Pi), your car can be self-driven as follow ...
 +
 
 +
* python manage.py drive --model ~/ohmc_car/models/models/model_$DATE.hdf5
 +
 
 +
This works similar to the manual driving mode with the addition of a trained model that can either ...
 +
 
 +
1) User: Manual control of both steering and throttle
 +
 
 +
2) Local Angle: Automatically control the steering angle
 +
 
 +
3) Local pilot: Automatically control both the steering angle and throttle amount
 +
 
 +
The web browser provides a drop-down menu to select between these options.
 +
 
 +
It is recommended to just start with Local Angle ... and control the throttle manually with the "i" key (faster) and "k" key (slower).
 +
 
 +
 
 +
== Neopixels ==
 +
 
 +
On the car, do this: pip install rpi_ws281x adafruit-circuitpython-neopixel  --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host www.piwheels.org
 +
 
 +
The details come from here: https://learn.adafruit.com/neopixels-on-raspberry-pi/overview
 +
 
 +
<pre>
 +
(env) pi@ohmc_24:~/ohmc_car$ diff -u manage.py.orig manage.py
 +
--- manage.py.orig 2018-06-11 08:14:34.079999106 +0000
 +
+++ manage.py 2018-06-11 08:37:05.959999560 +0000
 +
@@ -122,6 +122,9 @@
 +
    tub = TubWriter(path=cfg.TUB_PATH, inputs=inputs, types=types)
 +
    V.add(tub, inputs=inputs, run_condition='recording')
 +
 +
+    pixels[0] = (32, 64, 128)
 +
+    pixels[1] = (0, 255, 0)
 +
+
 +
    # run the vehicle
 +
    V.start(rate_hz=cfg.DRIVE_LOOP_HZ,
 +
            max_loop_count=cfg.MAX_LOOPS)
 +
@@ -169,6 +172,13 @@
 +
    cfg = dk.load_config()
 +
 +
    if args['drive']:
 +
+        # Neopixels
 +
+        # https://learn.adafruit.com/neopixels-on-raspberry-pi/python-usage
 +
+        import board
 +
+        import neopixel
 +
+        pixels = neopixel.NeoPixel(board.D18, 2)
 +
+        pixels[0] = (0, 0, 0)
 +
+        pixels[1] = (0, 64, 0)
 +
        drive(cfg, model_path=args['--model'], use_chaos=args['--chaos'])
 +
</pre>
 +
 
 +
Then, you need to run as root for neopixels to work:
 +
<pre>
 +
sudo /home/pi/env/bin/python manage.py drive
 +
--model=/home/pi/ohmc_car/models/large_track.hdf5
 +
</pre>
 +
 
 +
== Neopixels v2 ==
 +
 
 +
* Get https://gist.github.com/marcmerlin/a3bba2231a3371266e595aa0e6028b77
 +
* Save under /home/pi/ and /etc/rc.local
 +
* Apply manage.py.diff to /home/pi/ohmc_car/manage.py
 +
* Reboot and both the led server and the car software should start
 +
* You can force an led pattern or color like so
 +
<pre>
 +
(env) pi@ohmc_24:~/ohmc_car$ echo -n rainbow > /tmp/leds
 +
(env) pi@ohmc_24:~/ohmc_car$ echo -n b > /tmp/leds
 +
(env) pi@ohmc_24:~/ohmc_car$ echo -n cylon > /tmp/leds
 +
</pre>
 +
* Reboot and both the led server

Latest revision as of 04:14, 24 January 2019

Overview

From a software perspective, the DonkeyCar (Raspberry Pi) is self-contained ... driving, data acquisition (for training) and ultimately self-driving are all performed with on-board software. The provided micro-SD card already has all the required software pre-installed, as well as two pre-training A.I / Machine Learning models. The DonkeyCar software includes a web-server that provides a web interface that works on both desktop and mobile web browsers (great for driving).

Your laptop will be required to perform training of the Neural Network ... using the data acquired on the DonkeyCar. This means that Python software will need to be installed, e.g TensorFlow (Machine Learning framework).

Software installation: Laptop / Desktop

Extensive DonkeyCar documentation

  • apt-get install -y virtualenv # Note: You may be using a different software installer
  • mkvirtualenv donkeycar -p python3
  • pip install tensorflow==1.8.0 # Note: Probably requires Python 3.5 or 3.6. People are having problems with Python 3.7

(if you get errors, you can try (re-) installing pip: python -m pip install --upgrade pip )

On debian, if virtualenv isn't there, try this instead

  • virtualenv donkeycar -p python3
  • cd donkeycar
  • export PATH=`pwd`/bin:$PATH
  • pip install tensorflow==1.8.0

Then proceed:

Note: You can manually install Python 3.6 as follows, but then you can't use mkvirtualenv ...

  • yum install python36
  • python3.6 -m venv ~/virtualenvs/donkeycar
  • source ~/virtualenvs/donkeycar/bin/activate

Software installation: Raspberry Pi

Your DonkeyCar (Raspberry Pi) is already pre-installed. This section is for reference only.

Extensive DonkeyCar documentation

  • sysctl -w net.ipv6.conf.all.disable_ipv6=1 # Reconnect via ssh afterwards
  • sysctl -w net.ipv6.conf.default.disable_ipv6=1
  • apt-get update
  • apt-get upgrade
  • apt-get install -y vim git mosquitto-clients
  • apt-get install -y virtualenv build-essential python3-dev gfortran libhdf5-dev libatlas-base-dev libopenjp2-7-dev libtiff5
  • apt-get install -y i2c-tools
  • i2cdetect -y 1
        0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
   00:          -- -- -- -- -- -- -- -- -- -- -- -- --
   10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   40: 40 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   70: 70 -- -- -- -- -- -- --
  • mv env env.donkeycar
  • virtualenv env -p python3
  • pip install tensorflow==1.8.0
  • pip install adafruit-pca9685
  • pip install picamera

Finding your car on the network

Your DonkeyCar (Raspberry Pi) has been pre-configured to connect to the LCA2019 network. But, which IP address ?

Every car has a unique hostname, from ohmc_01 to ohmc_32. The micro-SD card adapter is labeled with your car name.

However, that still isn't enough to find your car.

So, there is a cron job that every minute sends an MQTT message to test.mosquitto.org with your car's name, IP address and a timestamp.

Install an MQTT client, as follows ...

  • apt-get install mosquitto-clients

Use the following command to read the MQTT messages ...

  • mosquitto_sub -h test.mosquitto.org -t 'ohmc/#' -v
 ohmc/ohmc_01 10.193.2.69 Mon 11 Jun 07:08:01 UTC 2018

Driving the car manually

Caution: Put your car "on blocks" (wheels off the ground) the first time you try driving it

Extensive DonkeyCar documentation

IP_ADDRESS=xxx.xxx.xxx.xxx # Found as above

  • ssh pi@$IP_ADDRESS # Note: Raspberry Pi default username: pi and password: raspberry
  • cd ohmc_car
  • python manage.py drive
 loading config file: /home/pi/play/roba_car/config.py
 config loaded
 PiCamera loaded.. .warming camera
 Starting Donkey Server...
 You can now go to http://xxx.xxx.xxx.xxx:8887 to drive your car.

With a desktop web browser, the user interface provides a virtual joystick (right-hand frame) that you can use to drive the car ... altering the steering and throttle values.

The mobile web browser, the user interface allows you to drive by tilting the phone left-right for steering and forwards-backwards for throttle. For safety, you must press the [Start Vehicle] / [Stop Vehicle] toggle button to enable control.

rPi reboots and Auto-starting donkeycar without logging in

If your rPi reboots when you're driving, check the following things:

  1. did you swap the servo wires between steering and power. Turn the ESC off, unplug one servo wire, turn the ESC back on (little toggle switch), and test steering. Make sure that steering does not activate the wheel motors.
  2. is your battery low? are you accelerating too quickly?

Because the battery has a high internal resistance, if you demand a lot of power too quickly, the voltage on the battery drops enough that the rPi gets below its critical voltage (when you add the voltage drop from the voltage converter), and the rPi reboots.

To restart donkey server automatically, do the following:

  • Add this to /etc/rc.local:

su - pi -c 'cd ~/ohmc_car; python manage.py drive &>/tmp/out' &

  • move 'source ~/env/bin/activate' from ~/.bashrc to ~/.profile

Steering and throttle calibration

To save time at the workshop, you won't need to calibrate your car's steering and/or throttle.

However, you may get better results and can perform calibration when you have time ... by following the DonkeyCar instructions

Data acquisition for the Neural Network

Once you are driving your car confidently around a track ... it is time to acquire training data for the Neural Network. DonkeyCar operates at 10 frames per second, capturing a 160x120 image, along with steering angle and throttle value. This is all stored in the $HOME/ohmc_car/tub/ directory.

Before training, it is a good idea to clean out previous data. Don't just remove all the files in the tub/ directory ! The tub/meta.json file is important.

Extensive DonkeyCar documentation

Perform the same commands as for manual driving ...

  • ssh pi@$IP_ADDRESS
  • cd ohmc_car
  • python manage.py drive

Then via the web browser press the [Start Recording] button ... drive the car around a track ... then press the [Stop Recording] button.

It is recommended that you collect between 5K and 20K frames. At 10 frames per second, that is between 500 and 2,000 seconds of driving. Make sure that you drive clockwise and anti-clockwise.

You will need to type this command just once on your DonkeyCar (Raspberry Pi)to provide a directory on your laptop for your training data ...

  • rsync -av pi@<car_ip>:ohmc_car .

When finished acquisition, then transfer the data from the DonkeyCar (Raspberry Pi) to your laptop / desktop for training the Neural Network.

  • rsync -av pi@<car_ip>:ohmc_car/tub ohmc_car/tub_$DATE

Training the Neural Network

Once training data has been copied to your laptop / desktop, you can begin training the Neural Network.

Extensive DonkeyCar documentation

Run these commands on your laptop / desktop to train the Neural Network ...

  • workon donkeycar # For those who have set-up a virtualenv
  • cd ohmc_car
  • python manage.py train --tub $HOME/ohmc_car/tub_$DATE --model ./models/model_$DATE.hdf5
 using donkey version: 2.5.7 ...
 loading config file: /Users/andyg/play/ai/roba_car/config.py
 config loaded
 tub_names ./tub_2019-01-15c
 train: 5740, validation: 1436
 steps_per_epoch 44
 Epoch 1/100
 2019-01-21 13:08:49.507048: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
 43/44 [============================>.] - ETA: 0s - loss: 58.5130 - angle_out_loss: 30.3421 - throttle_out_loss: 86.6839      
 Epoch 00001: val_loss improved from inf to 0.19699, saving model to ./models/roba0_2019-01-16c.hdf5
 44/44 [==============================] - 38s 874ms/step - loss: 57.1887 - angle_out_loss: 29.6601 - throttle_out_loss: 84.7172 - val_loss: 0.1970 - val_angle_out_loss: 0.3230 - val_throttle_out_loss: 0.0710

On a modern laptop, each epoch will take around 30 seconds to complete. For up-to 100 epochs. Typically, you can expect around 20 to 40 epochs before the Neural Network stop learning. That is around 10 to 20 minutes of training time.

The training command creates the Neural Network weights that represent what your DonkeyCar has "learned".

When training has completed, copy the trained model back to your DonkeyCar (Raspberry Pi)

  • scp $USERNAME@$HOSTNAME:ohmc_car/models/model_$DATE.hdf5 models

Self-driving ... the ultimate goal

Once your trained model has been copied back onto the DonkeyCar (Raspberry Pi), your car can be self-driven as follow ...

  • python manage.py drive --model ~/ohmc_car/models/models/model_$DATE.hdf5

This works similar to the manual driving mode with the addition of a trained model that can either ...

1) User: Manual control of both steering and throttle

2) Local Angle: Automatically control the steering angle

3) Local pilot: Automatically control both the steering angle and throttle amount

The web browser provides a drop-down menu to select between these options.

It is recommended to just start with Local Angle ... and control the throttle manually with the "i" key (faster) and "k" key (slower).


Neopixels

On the car, do this: pip install rpi_ws281x adafruit-circuitpython-neopixel --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host www.piwheels.org

The details come from here: https://learn.adafruit.com/neopixels-on-raspberry-pi/overview 

(env) pi@ohmc_24:~/ohmc_car$ diff -u manage.py.orig manage.py
--- manage.py.orig	2018-06-11 08:14:34.079999106 +0000
+++ manage.py	2018-06-11 08:37:05.959999560 +0000
@@ -122,6 +122,9 @@
     tub = TubWriter(path=cfg.TUB_PATH, inputs=inputs, types=types)
     V.add(tub, inputs=inputs, run_condition='recording')
 
+    pixels[0] = (32, 64, 128)
+    pixels[1] = (0, 255, 0)
+
     # run the vehicle
     V.start(rate_hz=cfg.DRIVE_LOOP_HZ,
             max_loop_count=cfg.MAX_LOOPS)
@@ -169,6 +172,13 @@
     cfg = dk.load_config()
 
     if args['drive']:
+        # Neopixels
+        # https://learn.adafruit.com/neopixels-on-raspberry-pi/python-usage
+        import board
+        import neopixel
+        pixels = neopixel.NeoPixel(board.D18, 2)
+        pixels[0] = (0, 0, 0)
+        pixels[1] = (0, 64, 0)
         drive(cfg, model_path=args['--model'], use_chaos=args['--chaos'])

Then, you need to run as root for neopixels to work: 

sudo /home/pi/env/bin/python manage.py drive 
--model=/home/pi/ohmc_car/models/large_track.hdf5

Neopixels v2

(env) pi@ohmc_24:~/ohmc_car$ echo -n rainbow > /tmp/leds
(env) pi@ohmc_24:~/ohmc_car$ echo -n b > /tmp/leds
(env) pi@ohmc_24:~/ohmc_car$ echo -n cylon > /tmp/leds
  • Reboot and both the led server
Retrieved from "http://www.openhardwareconf.org/index.php?title=OHMC2019_Software_instructions&oldid=967"