Difference between revisions of "OHMC2020 Software instructions"

From Open Hardware Miniconf
Jump to: navigation, search
m (killed WSL. Started with info about google colab)
m
Line 1: Line 1:
 
== Overview ==
 
== Overview ==
  
You will need to bring your own laptop to the assembly session.
+
You will need to bring your own laptop to the assembly session, ideally with a few tools preinstalled so you can maximise your time in the workshop building and playing, rather than installing and configuring.
 
 
To make the most of your time in the assembly workshop, it's best to install some tooling so you can maximise your time spent building and playing, rather than installing and configuring.
 
  
 
Why do we need a laptop?
 
Why do we need a laptop?
Line 9: Line 7:
 
From a software perspective, the DingoCar (Raspberry Pi) is self-contained ... driving, data acquisition (for training) and ultimately self-driving are all performed with on-board software.  The provided micro-SD card already has all the required software pre-installed, as well as two pre-training A.I / Machine Learning models.  The DingoCar software includes a web-server that provides a web interface that works on both desktop and mobile web browsers (which great for driving).
 
From a software perspective, the DingoCar (Raspberry Pi) is self-contained ... driving, data acquisition (for training) and ultimately self-driving are all performed with on-board software.  The provided micro-SD card already has all the required software pre-installed, as well as two pre-training A.I / Machine Learning models.  The DingoCar software includes a web-server that provides a web interface that works on both desktop and mobile web browsers (which great for driving).
  
But you need a laptop to train the Neural Network ... using the data acquired on the DingoCar.  To do this, you need Python and [https://www.tensorflow.org/lite/ TensorFlow] (a Machine Learning framework) on your laptop.
+
But you need a laptop to train the Neural Network ... using the data acquired on the DingoCar.   
  
 
== Software environment: Laptop / Desktop ==
 
== Software environment: Laptop / Desktop ==
  
For easiest, and fastest operation, you can use software on Google's Colab to process your car data in the cloud. This means all you need to have on your computer is:
+
For easiest, and fastest operation, you can use software on Google's Colab to process your car data in the cloud. This means you need to have on your computer:
  
 
* ssh: to connect to your car. (Windows users, it's now an [https://www.howtogeek.com/336775/how-to-enable-and-use-windows-10s-built-in-ssh-commands/ optional Microsoft update] or [https://putty.org/ install putty]).
 
* ssh: to connect to your car. (Windows users, it's now an [https://www.howtogeek.com/336775/how-to-enable-and-use-windows-10s-built-in-ssh-commands/ optional Microsoft update] or [https://putty.org/ install putty]).
Line 19: Line 17:
 
* a google drive account
 
* a google drive account
  
Google Colab instructions (make this more explicit)
+
'''Google Colab instructions'''
 
 
This is preferred because it's fast, doesn't require a big setup time on your local computer, and gives all workshop participants more time playing with their car and less time wrestling with installation and configuration issues. Also, if you don't have a GPU, it's significantly faster using Google's computer power to generate the model than it is to do it on your own.
 
 
 
* Go here once logged into to your google account. https://colab.research.google.com/github/robocarstore/donkey-car-training-on-google-colab/blob/master/Donkey_Car_Training_using_Google_Colab.ipynb
 
* Click the 'copy to drive' to make your own copy.
 
* Fill in the bits in the main area, step by step to follow the instructions to get the data from your car up to google, trained on google, then copy the results back down to your car. LIKE MAGIC
 
 
 
DIY instructions for when you don't/can't use Google Colab (move this to separate page)
 
  
We don't want Python 2, and people have reported problems with Python 3.7 or later. So we currently use Python 3.6.
+
This is preferred because it's fast, doesn't require a big setup time on your local computer, and gives you more time playing with your car and less time dealing with installation and configuration. Also, if you don't have a GPU, it's significantly faster using Google's computer power to generate the model than it is to do it on your own.
  
Miniconda instructions:
+
* Go to the [https://colab.research.google.com/github/robocarstore/donkey-car-training-on-google-colab/blob/master/Donkey_Car_Training_using_Google_Colab.ipynb google colab site] once logged into to your google account.
 +
* Click the 'copy to drive' to make a copy for your own use.
  
* Go to the [https://repo.continuum.io/miniconda/ Miniconda archive]
+
The left hand panel has information, the right hand panel is where the operations happen. It's a sequence of steps that you can run, and modify. The sequence of instructions walks you through uploading your data from your car to your google drive account, then running it through the colab machine learning model generator.
* Download Miniconda3-4.5.4 in the right system for you.
 
* In your command line prompt, go to the directory holding the file
 
* Run the script: ./Miniconda3-4.5.4-Linux-x86_64.sh (or equivalent)
 
  
This will by default add the Miniconda directory to your path. Now you can check you have Python 3.6 installed and available:
+
'''Local installation instructions'''
* python3 -i
 
* This should show you Python 3.6.5 | Anaconda Inc.
 
* Use quit() to get out of the python shell
 
  
'''Get Dingocar'''
+
Want to set up an environment on your own machine? Check out our [[quickstart guide]], or use the detailed and thorough instructions on the [https://docs.donkeycar.com/ DonkeyCar website].
 
 
Go to which directory you like to keep your coding projects in.
 
* git clone https://github.com/tall-josh/dingocar.git
 
* cd dingocar
 
* git checkout master
 
 
 
'''Install Tensorflow for machine learning'''
 
 
 
* Ubuntu
 
** apt-get install -y virtualenv  # Note: You may be using a different software installer
 
** mkvirtualenv donkeycar -p python3
 
** pip install tensorflow==1.8.0  # Note: Probably requires Python 3.5 or 3.6.  People are having problems with Python 3.7
 
(if you get errors, you can try (re-) installing pip: python -m pip install --upgrade pip )
 
 
 
* Debian, if virtualenv isn't there, try this instead
 
** virtualenv donkeycar -p python3
 
** cd donkeycar
 
** export PATH=`pwd`/bin:$PATH
 
** pip install tensorflow==1.8.0
 
 
 
(This seems to be v2, but 2019 instructions use 1.8?)
 
* conda install tensorflow-cpu
 
 
 
'''Install Dingocar'''
 
 
 
* pip install -e ./dingocar
 
 
 
'''Create an instance for your specific car'''
 
 
 
* donkey createcar --path ~/mycar #give your car its own unique name here!
 
  
 
== Find your car ==
 
== Find your car ==
Line 133: Line 88:
 
It is recommended that you collect between 5K and 20K frames.  At 10 frames per second, that is between 500 and 2,000 seconds of driving.  Make sure that you drive clockwise and anti-clockwise.
 
It is recommended that you collect between 5K and 20K frames.  At 10 frames per second, that is between 500 and 2,000 seconds of driving.  Make sure that you drive clockwise and anti-clockwise.
  
You will need to type this command just once on your DingoCar to provide a directory on your laptop for your training data ...
+
You will need to type this command just once on your DingoCar to provide a directory on your laptop for your training data:
  
 
* rsync -av pi@<car_ip>:ohmc_car .
 
* rsync -av pi@<car_ip>:ohmc_car .
Line 139: Line 94:
 
When finished acquisition, then transfer the data from the DingoCar to your laptop / desktop for training the Neural Network.
 
When finished acquisition, then transfer the data from the DingoCar to your laptop / desktop for training the Neural Network.
  
* rsync -av pi@<car_ip>:ohmc_car/tub ohmc_car/tub_$DATE
+
* First ssh into your car and create a tarball of your data for easier transfer
 
+
** tar czvpf tub_$DATE ohmc_car/data/tub_$DATE/
Once training data has been copied to your laptop / desktop, you can begin training the Neural Network.
+
* Then, on your computer, copy the data back to your computer.
 
+
** rsync -av pi@<car_ip>:ohmc_car/tub ohmc_car/tub_$DATE
[http://docs.donkeycar.com/guide/train_autopilot Extensive DonkeyCar documentation]
 
 
 
Run these commands on your laptop / desktop to train the Neural Network ...
 
 
 
* workon donkeycar  # For those who have set-up a virtualenv
 
* cd ohmc_car
 
* python manage.py train --tub $HOME/ohmc_car/tub_$DATE --model ./models/model_$DATE.hdf5
 
 
 
  using donkey version: 2.5.7 ...
 
  loading config file: /Users/andyg/play/ai/roba_car/config.py
 
  config loaded
 
  tub_names ./tub_2019-01-15c
 
  train: 5740, validation: 1436
 
  steps_per_epoch 44
 
  Epoch 1/100
 
  2019-01-21 13:08:49.507048: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
 
  43/44 [============================>.] - ETA: 0s - loss: 58.5130 - angle_out_loss: 30.3421 - throttle_out_loss: 86.6839     
 
  Epoch 00001: val_loss improved from inf to 0.19699, saving model to ./models/roba0_2019-01-16c.hdf5
 
  44/44 [==============================] - 38s 874ms/step - loss: 57.1887 - angle_out_loss: 29.6601 - throttle_out_loss: 84.7172 - val_loss: 0.1970 - val_angle_out_loss: 0.3230 - val_throttle_out_loss: 0.0710
 
 
 
On a modern laptop, each epoch will take around 30 seconds to complete.  For up-to 100 epochs.  Typically, you can expect around 20 to 40 epochs before the Neural Network stop learning.  That is around 10 to 20 minutes of training time.
 
  
The training command creates the Neural Network weights that represent what your DingoCar has "learned".
+
Once training data has been copied to your laptop / desktop, you can begin training the Neural Network. You can train your data on Google Colab, or on your local environment.  
  
 
Once it's done, copy the trained model back to your DingoCar (Raspberry Pi).
 
Once it's done, copy the trained model back to your DingoCar (Raspberry Pi).
Line 247: Line 181:
 
[http://docs.donkeycar.com/guide/calibrate DonkeyCar instructions on calibration]
 
[http://docs.donkeycar.com/guide/calibrate DonkeyCar instructions on calibration]
  
== OLED digital display ==
+
== What next? ==
  
Something here about image/text display for the car info.
+
A DingoCar is for life, not just for the OHMC at LCA. There's so much more you can do with it. Check out our [[Beyond DingoCar]] section.

Revision as of 11:40, 28 December 2019

Overview

You will need to bring your own laptop to the assembly session, ideally with a few tools preinstalled so you can maximise your time in the workshop building and playing, rather than installing and configuring.

Why do we need a laptop?

From a software perspective, the DingoCar (Raspberry Pi) is self-contained ... driving, data acquisition (for training) and ultimately self-driving are all performed with on-board software. The provided micro-SD card already has all the required software pre-installed, as well as two pre-training A.I / Machine Learning models. The DingoCar software includes a web-server that provides a web interface that works on both desktop and mobile web browsers (which great for driving).

But you need a laptop to train the Neural Network ... using the data acquired on the DingoCar.

Software environment: Laptop / Desktop

For easiest, and fastest operation, you can use software on Google's Colab to process your car data in the cloud. This means you need to have on your computer:

  • ssh: to connect to your car. (Windows users, it's now an optional Microsoft update or install putty).
  • scp: to copy files to and from your car (Windows users: putty comes with pscp which is equivalent)
  • a google drive account

Google Colab instructions

This is preferred because it's fast, doesn't require a big setup time on your local computer, and gives you more time playing with your car and less time dealing with installation and configuration. Also, if you don't have a GPU, it's significantly faster using Google's computer power to generate the model than it is to do it on your own.

  • Go to the google colab site once logged into to your google account.
  • Click the 'copy to drive' to make a copy for your own use.

The left hand panel has information, the right hand panel is where the operations happen. It's a sequence of steps that you can run, and modify. The sequence of instructions walks you through uploading your data from your car to your google drive account, then running it through the colab machine learning model generator.

Local installation instructions

Want to set up an environment on your own machine? Check out our quickstart guide, or use the detailed and thorough instructions on the DonkeyCar website.

Find your car

Every car has a unique hostname, from ohmc_01 to ohmc_32. The micro-SD card adapter is labeled with your car name.

Your DingoCar has been pre-configured to connect to the LCA2020 network. But how do we find out its IP address? There is a cron job that every minute sends an MQTT message to test.mosquitto.org with your car's name, IP address and a timestamp.

Install an MQTT client, as follows ...

  • apt-get install mosquitto-clients

Use the following command to read the MQTT messages ...

  • mosquitto_sub -h test.mosquitto.org -t 'ohmc/#' -v
 ohmc/ohmc_01 10.193.2.69 Mon 11 Jun 07:08:01 UTC 2018

The IP address returned will tell you how to connect to your car. Now you can:

``ssh pi@<IP_ADDRESS>`` with the default password '`raspberry'`. Change this once you log in, using ``passwd`` command.

Driving your car manually

Caution: Put your car "on blocks" (wheels off the ground) the first time you try driving it

  • ssh pi@<IP_ADDRESS>
  • cd ~/mycar
  • python manage.py drive
 loading config file: /home/pi/play/roba_car/config.py
 config loaded
 PiCamera loaded.. .warming camera
 Starting Donkey Server...
 You can now go to http://xxx.xxx.xxx.xxx:8887 to drive your car.

With a desktop web browser, the user interface provides a virtual joystick (right-hand frame) that you can use to drive the car ... altering the steering and throttle values.

The mobile web browser, the user interface allows you to drive by tilting the phone left-right for steering and forwards-backwards for throttle. For safety, you must press the [Start Vehicle] / [Stop Vehicle] toggle button to enable control.

Further DonkeyCar docs on driving

Training your car

Once you are driving your car confidently around a track, it is time to acquire training data for the Neural Network. DingoCar operates at 10 frames per second, capturing a 160x120 image, along with steering angle and throttle value. This is all stored in the $HOME/ohmc_car/tub/ directory.

Before training, clean out previous data. Don't remove all the files in the tub/ directory: the tub/meta.json file is important.

Perform the same commands as for manual driving:

  • ssh pi@$IP_ADDRESS
  • cd ohmc_car
  • python manage.py drive

Then via the web browser press the [Start Recording] button, drive the car around a track, then press the [Stop Recording] button.

It is recommended that you collect between 5K and 20K frames. At 10 frames per second, that is between 500 and 2,000 seconds of driving. Make sure that you drive clockwise and anti-clockwise.

You will need to type this command just once on your DingoCar to provide a directory on your laptop for your training data:

  • rsync -av pi@<car_ip>:ohmc_car .

When finished acquisition, then transfer the data from the DingoCar to your laptop / desktop for training the Neural Network.

  • First ssh into your car and create a tarball of your data for easier transfer
    • tar czvpf tub_$DATE ohmc_car/data/tub_$DATE/
  • Then, on your computer, copy the data back to your computer.
    • rsync -av pi@<car_ip>:ohmc_car/tub ohmc_car/tub_$DATE

Once training data has been copied to your laptop / desktop, you can begin training the Neural Network. You can train your data on Google Colab, or on your local environment.

Once it's done, copy the trained model back to your DingoCar (Raspberry Pi).

  • scp $USERNAME@$HOSTNAME:ohmc_car/models/model_$DATE.hdf5 models

Further DonkeyCar info on training

Letting your car drive itself!

Once your trained model has been copied back onto the DingoCar, your car can be self-driven as follows:

  • python manage.py drive --model ~/ohmc_car/models/models/model_$DATE.hdf5

This works similar to the manual driving mode with the addition of a trained model that can either:

1) User: Manual control of both steering and throttle 2) Local Angle: Automatically control the steering angle 3) Local Pilot: Automatically control both the steering angle and throttle amount

The web browser provides a drop-down menu to select between these options.

It is recommended to just start with "Local Angle" and control the throttle manually with the "i" key (faster) and "k" key (slower).

More information, for later

There's extensive DonkeyCar documentation if you're looking for more detailed installation or configuration instructions.


Background information

Software installation: Raspberry Pi

Your DonkeyCar (Raspberry Pi) is already pre-installed. This section is for reference only.

Extensive DonkeyCar documentation

  • sysctl -w net.ipv6.conf.all.disable_ipv6=1 # Reconnect via ssh afterwards
  • sysctl -w net.ipv6.conf.default.disable_ipv6=1
  • apt-get update
  • apt-get upgrade
  • apt-get install -y vim git mosquitto-clients
  • apt-get install -y virtualenv build-essential python3-dev gfortran libhdf5-dev libatlas-base-dev libopenjp2-7-dev libtiff5
  • apt-get install -y i2c-tools
  • i2cdetect -y 1
        0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
   00:          -- -- -- -- -- -- -- -- -- -- -- -- --
   10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   40: 40 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   70: 70 -- -- -- -- -- -- --
  • mv env env.donkeycar
  • virtualenv env -p python3
  • pip install tensorflow==1.8.0
  • pip install adafruit-pca9685
  • pip install picamera

rPi reboots and Auto-starting donkeycar without logging in

If your rPi reboots when you're driving, check the following things:

  1. did you swap the servo wires between steering and power. Turn the ESC off, unplug one servo wire, turn the ESC back on (little toggle switch), and test steering. Make sure that steering does not activate the wheel motors.
  2. is your battery low? are you accelerating too quickly?

Because the battery has a high internal resistance, if you demand a lot of power too quickly, the voltage on the battery drops enough that the rPi gets below its critical voltage (when you add the voltage drop from the voltage converter), and the rPi reboots.

To restart donkey server automatically, do the following on your car:

  • Add this to /etc/rc.local:

su - pi -c 'cd ~/ohmc_car; python manage.py drive &>/tmp/out' &

  • move 'source ~/env/bin/activate' from ~/.bashrc to ~/.profile

Steering and throttle calibration

To save time at the workshop, you won't need to calibrate your car's steering and/or throttle. However, you may get better results and can perform calibration when you have time.

DonkeyCar instructions on calibration

What next?

A DingoCar is for life, not just for the OHMC at LCA. There's so much more you can do with it. Check out our Beyond DingoCar section.