Header Overview

The VI-Sensor is an advanced payload.
Please carefully read this section and always perform the described checks to make sure everything is working as intended.
Do not take off if any of the provided steps below report any errors!

Discontinued Payload

This payload is discontinued and will not experience further updates.

Prerequisites

This section describes how to use the VI-Sensor mounted on an AscTec Firefly in delivery condition.

Before we start, make sure you have the following:

Furthermore, before starting any experiments:

  • Go through the whole First steps section
  • Learn to safely fly the UAV outdoors in GPS mode and Height mode, if possible also Manual mode.
  • Leave the AscTec Mastermind switched OFF during your first steps and training maneuvers.

Setup

This How-To assumes that the user is familiar with the basic handling of a Ubuntu-based Linux.

Firefly + Mastermind

To successfully retrieve telemetry data from the Firefly, it needs to be configured to automatically join an available Wifi.
First of all, configure the hardware:

  1. Connect your Mastermind to a VGA monitor with the Mastermind's VGA adapter cable
  2. Connect a mouse and a keyboard to the Firefly (you will most probably need the mini-USB to USB adapter cables)
  3. Connect a battery to the Firefly
  4. Switch on the Mastermind

Now configure the software as follows:

  1. You will automatically be logged in as user "asctec". In case you need superuser rights, the sudo password is "asctec".
  2. Start the graphical desktop with "sudo start lightdm"
  3. In the top right of the desktop is the Network-Manager interface, (right-)click on it and select "Edit Connections"
  4. Go to wireless and select the Wifi of your access point
  5. Press edit and enter your Wifi credentials
  6. In the "IPv4 Settings" tab, choose method "Manual" and add an address. This How-To will use the address "192.168.1.100". The netmask is "24". Leave the Gateway field empty.
  7. In the "IPv6 Settings" tab, choose method "Ignore"

The Mastermind should now be properly configured and automatically join your Wifi. Please, shut it down (sudo init 0) for now, we will first configure your virtual machine.

Virtual Machine

Now, we will configure the provided Virtual Appliance. This virtual machine contains all the required software to verify that the VI-Sensor is working as intended. This includes a complete ROS installation with all the required software packages.
Start with importing the Virtual Appliance into Virtual Box:

  1. Start VirtualBox, go to File => Import Appliance
  2. Select the downloaded .ova file, click Next
  3. You can most probably just keep the provided settings. Eventually you may want to change the storage folder for the VM's harddisk (last entry in the settings).
  4. Click import, this process may take a few minutes

 

Depending on your network environment you may need to change some network settings of the Virtual Machine. Right-Click on the machine and select "Change", afterwards go to the Network tab.

Wired + Wireless Adapter

The PC used in this tutorial had one wired network interface with internet connection and one wireless interface connected to a test Wifi. If this is also the case for your setup do the following:

  1. Leave "Adapter 1" with its current settings
  2. Go to the "Adapter 2" tab
  3. Select "Network Bridge" and below that, select your wireless network card ("Name" drop-down)
  4. Please do NOT change any MAC address

Wired Adapter

If your PC only has a wired connection to the Access Point, please do the following:

  1. Disable "Adapter 1" by disabling the ckeckbox labelled "Networkadapter active"
  2. Go to the "Adapter 2" tab
  3. Select "Network Bridge" and below that, select your wired network card ("Name" drop-down)
  4. Please do NOT change any MAC address

Click "OK" to accept all changes. Now start the virtual machine. The "asctec" user will be automatically logged in. In case you need superuser rights, the sudo password is "asctec".

Now, configure the network settings of the virtual machine:

  1. Click on the Network-Manager icon in the top right and select "Edit connections"
  2. There are one or two wired interfaces, depending on your network configuration. The connection to your access point (may it be wired, or wireless) is always named "Wifi".
  3. If your access point has a DHCP service running, you do not need to configure anything
  4. If you don't have DHCP, please go to the "IPv4 Settings" tab, switch to method "Manual" and assign an IP address (e.g. 192.168.1.120).
  5. Click Save

Furthermore, we need to add a static name resolution from "mastermind" to it's IP address (remember that this How-To assumes it to be 192.168.1.100):

  1. Open a terminal emulator (e.g. Terminator). Move your mouse to the bottom of the virtual machine's screen to let a shortcut bar pop-up. Start either Terminator or Terminal Emulator.
  2. Edit the "/etc/hosts" file with the command "sudo mcedit /etc/hosts". Remember that the sudo password is "asctec".
  3. There is a line like "10.10.126.200 mastermind.local mastermind". Change the IP address to 192.168.1.100.
  4. Save (F2) and quit (F10) the editor.

Initialization

  1. Please put your Firefly on an even and stable surface.
  2. The position should ideally be a bit elevated (e.g. on a box), so that the cameras' field of view is filled with features
  3. In the direction of the camera should be a feature-rich environment, i.e. object edges and structure. A plain grey carpet does NOT have any features!
  4. Switch on the Firefly
  5. Switch on the Mastermind
  6. Proceed with the tests below.

The VI-Sensor needs more features during the initialization sequence than during normal operation. Once the initialization is complete, you may put the Firefly on the ground. It is usually sufficient if there are some features in the upper third of the camera images.

Pre-Flight Tests

Network

If you properly set up your network, the Wifi, and the Mastermind, you should now be able to ping the mastermind with:

This should output a reply from the Mastermind with the round-trip time. Hit CTRL+C to abort the ping.

ROS

To check if the ROS master node is running, you may use the nmap command:

This should output something like "11311/tcp open ...". If you don't get any response, or it mentions "closed", the startup of ROS is not fully done yet. Just wait a bit and repeat the test.

It takes approx. 2 minutes after powering on the Mastermind until a Wifi connection is established and ROS is fully operational.

Pose Estimation & Feedback

You can now start some tools to check if the VI-Sensor pose estimation is working properly. Move your mouse to the bottom of the screen, on the very right is a small satellite-like icon, click on it. This will start two programs and one important text output:

rqt_plot - Position Drift

A small graphical plot of three sensor values. The values must all be close to zero (+-0.02) when the Firefly is standing still. If there are larger values or the values change constantly, the initialization was not perfect. Switch off the Mastermind and restart it in a feature-rich position.

rviz - Pose Visualization & Camera Images

RViz will show you the position of the Firefly and the VI-Sensor. The point where the Firefly has been initialized is the origin. You can also see the output of the camera images to verify that there are sufficient features.
You may now take the Firefly in your hands and move it around. You will see the movements in RViz. This verifies that the VI-Sensor's state estimation and the complete processing chain is working properly. 

It´s normal if the output of the camera images is shown upside down. 

AutoPilot Feedback

In the active Terminator window, there is a status feedback from the AutoPilot in the lower right window. It is constantly updated and displays e.g. the current battery voltage.
Please do the following checks with the RC while having this window open:

  1. Switch on the RC
  2. Put the mode switch to manual (red switch away from you) and the serial switch off (black switch away from you)
    1. The AutoPilot feedback should report "Flight Mode: Acc" and "Serial: false"
  3. Put the mode switch in GPS mode (towards you) and serial switch still off
    1. The AutoPilot feedback should report "Flight Mode: GPS" and "Serial: false"
  4. Put the mode switch in GPS mode and the serial switch ON
    1. The AutoPilot feedback should report "Flight Mode: Acc" and "Serial: true"

If any of the checks does not output the expected result, there is a configuration problem with your RC.

Take-off

Please make sure all pre-flight tests were successful. Do not try to take off if there were any problems!

This section describes the take-off procedure and the actions required to enable the indoor-position hold.

  1. Put your Firefly gently on the ground and make sure that the pose estimation is still stable (does not drift away) by checking RViz
  2. Make sure you have enough space around you. A 5x5 meter area and a height of at least 2.5m is the minimum for your first try. If there is no wind and you can take the access point with you, you may also do this outdoors.
    Keep in mind that the limit of the VI-sensor where it can recognize features is approx. 15m, so make sure the VI-sensor always has sufficient features in sight. 
  3. Put the RC in manual mode (red switch) and serial OFF (black switch)
  4. Enable the motors (minimum thrust + full yaw command) and leave them at minimum thrust
  5. Still in manual mode, slowly move the thrust stick towards the center position, the motors will increase the speed, but the Firefly should not take off! If it does, slightly reduce the thrust.
    If you are not sure how much thrust is needed to let the Firefly take off, you could get a feeling for that by pushing the Firefly gently to the ground and increasing the thrust slowly. Please be extremely careful and do not touch the propellers. 
  6. You should now be at the thrust level where the Firefly barely NOT takes off.
  7. Now simultaneously switch directly to GPS mode (red switch towards you) and enable serial mode (black switch towards you). The Firefly should remain at the ground.
  8. Now slightly move the thrust stick above the center position. Wait a bit, after a delay, the Firefly will increase its thrust and take off. If it is at 1m, move the thrust stick back to center, it should now keep its height and position. The delay comes from the internal position controller, please be patient during your first tries and don't move to full thrust.
  9. You can now control the Firefly with the RC like in normal GPS mode.

Note that the controller is configured very softly. If you are putting the RC stick in center position while the Firefly has a horizontal velocity, the current velocity will decay slowly. The Firefly will not stop immediately!

During your first take-off tries, it can help to keep one hand on the Firefly to push it down. This gives you more confidence and you will get a feeling for the delay until the Firefly wants to take off.

Whenever you are handling the Firefly with running motors, we recommend to use protective gloves!

Landing

Landing is very simple. Just move the thrust stick down and the Firefly will decent. When it touches the ground you can reduce thrust to the minimum. Now switch off serial mode and go to manual mode. Finally, give a minumum thrust + full yaw command to switch of the motors.

Congratulations on your first flight with the VI-Sensor!

FAQ

Is a Wifi connection required to start up the VI-Sensor processes?

Yes, the automated start scripts on the Mastermind will not complete if there is no Wifi available.
You can adjust them later on, when you are familiar with the flight system and remove this barrier.

What happens if the Wifi connection is lost in-flight?

This is not a problem. You will not receive any further telemetry information, but the VI-Sensor and the positioning will still be fully functional. That is all done onboard of the UAV. The Wifi connection is only used for the tests.

How susceptible is the VI-Sensor to changing light conditions?

The VI-Sensor can handle such situations quite well, e.g. turning indoor from an artificial low-light situation to outdoor (sunlight) is well compensated by the automatic camera exposure.
If you are in a very dark room or the sun light might blind the sensor, please check the camera images, if there are sufficient visible features.

What happens in a structure-less envrionment?

As all vision-based sensors, the VI-Sensor requires some structure (altough very little) in its envrionment to estimate its position. E.g. a plain white wall will certainly lead to a drift after a few seconds.

What happens in a dynamic envrionment (e.g. walking by people)?

The VI-Sensor is clever enough to separate between static and dynamic objects. As long as there are some static features on the images, the pose estimation will work without any problems.

Does the VI-Sensor provide crash-protection (a.k.a. obstacle avoidance)?

No! The VI-Sensor with its current firmware does not provide any obstacle detection or avoidance. If you command the Firefly to fly forward into a wall, it will do so!