How to Set Up Your Python OpenCV Development Environment

For our drone flying project, we needed a way for our computer to detect the location of our mini-drone through the use of a webcam mounted above the flying area. We are not at all familiar with computer vision algorithms, but we do know how to call functions from a Python library! We made use of OpenCV (Open Source Computer Vision), which is available for Python and C++.

For our Python environment, we chose Python(x,y). Python(x,y) is a version of Python developed specifically for scientific calculations and visualizations. If you are a fan of Matlab, then you will feel right at home with Python(x,y).

This is what you need to do to set up a Python(x,y) development environment with OpenCV.

    1. Install the latest revision of the python(x,y) package. This includes Spyder (Scientific PYthon Development EnviRonment). Download Python(x,y) here.
    2. For the Python(x,y) install, choose Custom install and select the PythonPySerial 2.7-1 component. PySerial is needed to communicate with an Arduinopython install
    3. Optional: We also like to add the OtherWinMerge component when installing Python(x,y), but it is not required.
    4. You will also need to install the opencv2 package. Download opencv2 here.
    5. Unzip the opencv2 package and copy
      opencv\build\python\2.7\x86\cv2.pyd to <python dir>\Lib\site-packages\ where the default Windows location for <python dir> is C:\Python27

Note: If your computer supports it, copy opencv\build\python\2.7\x64\cv2.pyd instead of x86. I decided which to run by first trying the x64 copy, but the x64 version did not work for me when run. So I copied the x86 version instead. See below for how to check if OpenCV is loading properly.

Now it’s time to check if your development environment is working. Start Python(x,y) and you will see this window:

Spyder

Click on the small blue and red icon Spyder button that looks like a spider web to start the Spyder IDE. Here is what the Spyder IDE looks like:

Spyder IDE

The bottom right portion of the IDE shows the IPython console. You can run scripts or call Python commands directly in the IPython console.

In the IPython console, type import cv2 and hit enter.

If there is a problem, then you will receive an error, likely an error about “No module named cv2”. If that happens, then check that you copied the OpenCV files to the correct location as described in Step 3 above.

If everything is working, then the console will accept your command and show a prompt for your next command like this:

import cv2

Hooray, you have successfully set up Python(x,y) and OpenCV! Nothing to it, right? Now let’s see what we can do with OpenCV. Take a look at our post on blob detection with OpenCV.

Teach your PC to fly a Mini-Drone!

A few months ago, I watched this TED talk where they setup an indoor arena and did some amazing things with drones.  It got me thinking, and it inspired me to build something like that for myself – but on a much smaller and cheaper scale.

In the video they use an expensive real-time infrared motion tracking system (I am guessing something like these Optitrack systems) to measure the position of the drones, and then uses a computer to calculate and send control signals to coordinate the drones. At a high level, my setup works in a similar way, as shown in this diagram:

Here’s a photo of what my setup looks like:

drone setup
Photo of the first working setup.

This is a list of the items needed to build this:

Component Description
USB Camera ELP 2megapixel Hd Free Driver USB Camera Support Mjpeg Linux Android Windows Developing Board,usb Camera Module
Arduino Microcontroller Board Arduino UNO R3 Board Module With DIP ATmega328P(Blue)
Nordic Semiconductor 2.4GHz Wireless Card Addicore nRF24L01+ Wireless AddiKit with Socket Adapter Boards and Jumper Wires
Cheerson CX-10 Mini-Drone Cheerson CX-10 Mini 29mm Diameter 4CH 2.4GHz 6 Axis Gyro RC Quadcopter UFO RTF Green
2 blade guards for the Cheerson CX-10 Upgrade Cheerson Cx-10 Propeller Prop Blade Guard Cover Bumper Protection Protector Green White

Total cost for these items was around $85. In addition to the above, you might also need a folding table and stack of books to hold up the webcam as I did, but you can probably think up something more refined!
Here is a video of it working:

Here are some links to further information on how this all works:

  1. Setting up the programming environment on your PC
  2. Detecting the circles from the webcam
  3. Finding a low-latency web camera

Source code:

  1. nrf24_cx10_pc  – The source code for the Arduino to send 2.4GHz wireless signals to the drone
  2. pc-drone – The Python / OpenCV code used to track the drone and decide on how to adjust the drone controls

We will also be sharing this project at the Bay Area Maker Faire from May 20-22, so please stop by the MakeHardware.com booth and check it out!

What is an OTA DVR and why would I want one?

OTA DVR stands for Over-the-air Digital Video Recorder. Basically, this is a digital video recorder (think Tivo) system that allows you to record programs from over-the-air broadcast signals.

Long ago, before there was such a thing as cable or satellite TV, everyone watched TV by attaching antenna to the television set and pulling in a signal that transmitted by a rabbit ear antenna.

These signals are still being broadcast in most of the country, although the format and the quality are much improved.  Instead of fuzzy analog images like before, stations are now broadcasting in 720p or 1080i high definition digital signals.  In many case, the bandwidth and image quality is equal or even better than what your cable or satellite provides.  Plus the antennas don’t look as dorky anymore (hopefully you have a better TV too!)

Why Do I need a DVR if I have Streaming?

When I first cancelled my cable subscription (“cut the cord”) and setup an antenna way back in 2011, the main thing I missed was the DVR that was previously provided by the cable provider.  I could still get all the local broadcast networks (ABC, CBS, Fox, NBC, PBS). When ESPN became available through streaming on Sling TV (it’s also now available on Sony’s Vue), then I had access to pretty much everything I needed. However, I still missed my DVR because:

  1. I couldn’t skip commercials anymore!  Basically, if you’re streaming, they can force you to sit through as many commercials as they want.
  2. The skip-forward/skip backward functions are pretty crappy in every streaming app I’ve ever seen, especially compared to the responsiveness of a DVR.  I’d rather watch a recorded TV show from a broadcast channel versus from a streaming service just because of this.
  3. Not all streaming channels work that well, especially when watching live sports events.  There are glitches, dropouts, and sometimes the resolution gets lower (and I have 100Mbps internet service). At least in my case, the reliability of the streaming signal is not as good as the reliability of the antenna signal.  The Antenna signal is slightly less reliable than a cable signal, but the antenna is free, so I find it to be a good compromise.

Anyway, that’s my rant about why DVR’s are still nice to have. If you got this far, then you’re probably want to know how to get your own OTA DVR.  Checkout my guide here. 

 

Finding a low-latency webcam

When trying to use a webcam in a computer vision application as part of a real-time control system, the latency is often just as important as the frame rate. Unfortunately, the latency for a webcam is often not specified, especially not for low-cost webcams.

One simple way to measure the webcam latency is to point the camera at a computer screen that is displaying the view from the camera and also printing the current time on the screen.  You end up with infinite recursion images like this:

Latency Test Images
Measurement of ELP-USB500W02M-L21 webcam latency
Measurement of ELP 5mp webcam latency
Measurement of ELP-USBFHD01M-L36 webcam latency
Measurement of ELP USB high-speed webcam latency
Measurement of PS3 Eyecam Latency
Measurement of PS3 Eyecam Latency

The difference between the time which is overlaid on the image (the largest type) and the time shown in the image from the webcam (the next largest) is the latency.  The Python / OpenCV2 code I used to capture these screenshots is up on Github.

Here are the results for three cameras I measured:

Camera Model No. Latency (ms) Frame-Rate (fps)
ELP USB with Camera 2.1mm Wide Angle Mjpeg 5megapixel Hd Camera USB for Industrial, Machine Vision ELP-USB500W02M-L21  ~115 to ~130  10
ELP 2megapixel Hd Free Driver USB Camera Support Mjpeg Linux Android Windows Developing Board,usb Camera Module ELP-USBFHD01M-L36  ~105  30
PlayStation Eye PS3 Eyecam ~75  60

All cameras were set to capture at 640 x 480.  The above cameras are all consumer grade cameras, costing about $45 for the ELP models, and only $5 (!) for the PS3 Eyecam.  As a comparison point, $280 would get you the Slim-3U from Optitrack which is specifically designed for motion capture and has a 8.33ms latency. Let me know if you find any other sub-$100 cameras that perform better!

ELP 5 Mega-pixel USB camera with 2.1mm Wide Angle Lens

This camera has a nice image quality, but the frame rate is slow and the latency was inconsistent.  The lagginess of this camera is very evident upon first usage of the camera.

ELP 2megapixel USB Camera with 3.6mm lens

This camera had good image quality, tolerable latency, and a 30fps rate.  I tested the version with a 3.6mm lens, but the base camera model USBFHD01M is also available with a  170degree fisheye lens or 2.1mm lens.  There is a nice review of this camera here.

This is the camera that I ultimately chose for my computer vision project.

PlayStation Eyecam

For a cost of $5, this is a very interesting camera.  The latency of this camera was the most consistent, and it is also capable of higher frame rates.  In order to use it on a Windows system, you will want to purchase a driver from Code Laboratories at a cost of $2.99 (it works very well). One tip, you may need to create a cleye.config file and save it in “C:\Program Files\Code Laboratories\CL-Eye Driver” to get greater than 30fps from the camera/driver.  This file contains this text:

<?xml version=”1.0″ encoding=”UTF-8″?>
<cleye>
<item name=”mode” value=”advanced” />
</cleye>

The image quality from this camera was not that great though (you can see it’s image is more blurry than from the other two camera), so that is why I did not choose this camera for my project.  It would be great for applications where there is fast motion and image quality is not as critical.

Industrial cameras and other links

In addition to the Optitrack camera mentioned above, here are a few more cameras I found discussed on Reddit that offer low latency. These are higher cost industrial cameras:

This is a project that measured latency using the same technique shown here using a Pi camera on a Raspberry pi.

How to Get Your Boss to Buy You a Quadcopter

Every year my boss asks for suggestion for a teambuilding activity to have with his staff.  Past activities have included thrilling activities like bocce ball and an indoor trampoline park (tip: this is a great idea if your team consists of 8 year-olds). This year I suggested that he buy everyone some drones and that we run some drone races. My suggestion was rejected – we went bowling instead.
Well, I took matters into my own hands. I manage a team of engineers myself, and I was determined to do something a little different this year.

Step 1. Plan

Instead of the usual holiday lunch, I bought everyone on my staff a Cheerson CX-10 quadcopter.  These are tiny little things – only about 2.5″ across, and they’re great for flying around indoors.  Best of all, they only cost about $16 on Amazon. I gave them to everybody during a staff meeting, and scheduled another “meeting” about 3 weeks later to give everyone time to practice.  Encourage them to practice – most likely they’ll need it.


CHRSONX10-O_1
The original basic version of the Cheerson CX-10.
$16 on Amazon. 

Step 2. Execute

At this next “meeting,” we had two events on the agenda:

Obstacle course

We put 5 foam pads on the ground and on some tables.  Most of them were about 8″x 8″, but one was about 4″ x 4″ for an extra challenge.  The goal was to take off and land on each of the foam pads and return to the starting point.  A successful landing on the large landing pad earned 50 points or 100 points for the smaller one. We had a time limit of 3 minutes to complete the course.  Any crashes or other mishaps that require the pilot to touch the quadcopter (such as flipping it back upright) to were penalized with -30 points.

IMG_1156
Foam landing pad. 50 points!

Race

We setup a simple race course using a few poles as markers to make a loop.  We had heats with 4 quadcopters flying at a time.  Going around in a loop is harder than it looks.  In both heats, the winner was the one who managed to fly three laps without crashing.  Practice helps.

Repairs

During the 3 weeks of practice, about 1/4 of my team managed to damage their quadcopters, so make sure that you have some spare parts available. Refer them to this repair guide.

Step 3. Profit

In addition to having a fun afternoon, I got to introduce a bunch of my coworkers to flying quadcopters, and we still fly them around sometimes (when no one else is watching).

Don’t forget this final critical step: Record your expenses and submit them under the category “Internal Meeting / Meal / Entertainment.”

Next year I think we’ll scale things up and go for a larger quadcopter we can fly outside – maybe a Hubsan X4.

3D Printed Spaceship Blade Guards for the Cheerson CX-10

If you’ve got access to a 3D-printer, or are just looking for an excuse to use one, these little blade guards are a neat project.  Making your quadcopter look like a spaceship seems to be quite the thing, here’s links to some examples.

Starships_preview_featured
Starship and Millenium Falcon Propeller guards.  This Thingiverse project has designs for both the Hubsan Q4 and Cheerson CX-10 – make sure you choose the right one!
Millenium Falcon Bladeguard for Cheerson CX-10
Millenium Falcon Bladeguard for Cheerson CX-10. I think this may have been the original one, the others may have been variants.
Millenium Falcon Spaceship Cheerson CX-10 bladeguard Remix
Millenium Falcon Spaceship Cheerson CX-10 bladeguard Remix
Cheerson CX-10 Quadcopter Wheels
This project is called “quadcopter wheels”, but it could pass for a rudimentary Tie-Fighter too.

 

 

Cheerson CX-10 Repair Guide and Tips

For all of you who received Cheerson CX-10 mini-quadcopters as holiday gifts and are learning to fly, bookmark this page because you’ll need it soon.  Here’s a list of some common repairs – you might as well learn about it now.

FYI – The Cheerson CX-10 is also sometimes sold as the Cheerson Q4.  This repair guide also largely applies to the similar Cheerson CX-10A and Cheerson CX-10C models.

Also just for reference, here is the manual for the Cheerson CX-10.

0.05 Ounces of Prevention

If you received this as a gift:

Cheerson CX-10

and it didn’t come with a prop guard (which weighs 0.05 ounces / 1.5g), then you should definitely get one. It will dramatically reduce the number of broken propellers you have to deal with, particularly while you are first learning.

Blade Guard and 16 replacement propellers for Cheerson CX-10
2 blade guards for the Cheerson CX-10

Replacing the propellers on your CX-10

Removing the propellers: The propellers can be removed from the motor shaft by pulling them straight upwards.  Sometimes they are loose, but sometimes it can take a little force. A pair of wirecutters gently put around the prop can help pry off the more stubborn ones.  Just take care not to bend the motor shaft or nick it.

wirecutter for propeller removal

Proper Propeller Placement: You will need to remove the propellers when they break, but even more often something will get tangled in them (in my house, that tends to be hair or carpet fuzz). Removing the propeller makes it easy to remove the offending tangle.

Cheerson CX-10 propeller placement
Arrows indicate motor rotation direction. Make sure you put the propeller on so that when it rotates it pulls the quadcopter up!

The key thing to remember is that when you put the propellers back, you need to match the direction of rotation with the propeller.  There are two types of propellers- some of them rotate clockwise, and some rotate counterclockwise. There is actually a letter code marking the propellers, but it’s easy enough to just remember the direction of rotation of the motor, and make sure that when the propeller rotates it should be pulling the quadcopter upwards.

Shorted out motor wires

This is a common issue. On one of my quadcopters, the motor wiring was worn and exposed on all four motors.  This is what it looks like:

shorted motor wire on Cheerson CX-10IMG_1066-zoom
Worn insulation and exposed wiring on Cheerson CX-10 motor

The problem is that the wiring will sometimes short against the metal body of the motor.

In my case, the quadcopter would still turn on and pair with the controller, but then once you tried to fly it the motors would spin for a half-second and then stop.  The LEDs would then start blinking like it need to be charged again (even though it had just been charged).

On another quadcopter, it would fly for a few seconds, and then fall out of the sky.  Amusing, but frustrating.

Here is the fix:

  1. Remove the propellers
  2. Unscrew the four screw in the bottom.
  3. Gently pry off the the bottom white enclosure (note the clips on each end of the PCB)
  4. Take off the blue case

You should end up with something that looks like this:

IMG_1062
CX-10 parts disassembled
IMG_1063
CX-10 PCB, motors, and battery.

5. Fixing the insulation on the worn motors is tricky because it might make it too thick to slide the motor back into the case.  In one case, I just removed the motor and remounted it so that the exposed part is away from the motor body.  In another case I used tweezers to put in a place a very small piece of black electrical tape to insulate the wire.

6. Put everything back together (be careful when replacing the propellers!)

Some other posts about the CX-10

The Cheerson CX-10, CX-10A , CX-10C, CX-10D, CX-10WD, CX-10WF and Related Mini-Quadcopters

There are a few variations on the Cheerson CX-10, and it can be a little confusing as to what the differences are.  Here is a quick guide.

Cheerson CX-10 The original
Cheerson CX-10A Adds headless mode
Cheerson CX-10C Adds a camera that can take video or still pictures and save them to an onboard SD card.
Cheerson CX-10D This version adds an altitude hold capability that can make it easier to fly.  This is made possible by a very sensitive barometric (air pressure) sensor.  For some reason it is mainly sold in some hideous rainbow or camouflage colors!
Cheerson CX-10W This amazing version adds first-person view capability (FPV). It has an on board camera the transmits a live image to your iOS or Android phone, which is also used as the controller.
Cheerson CX-10WD This version has the FPV capabilities of the CX-10W and adds the altitude control of the CX-10D.
Fluoreon FX-10 This is a slightly smaller version of the CX-10A (it also has headless mode)

See below for more information on all of these.

 

The Original: Cheerson CX-10

cx-10 The original basic version of the Cheerson CX-10.

Check price on Amazon. 

For those interested in hacking the CX-10, it looks like there has been at least 3 different PCB revisions. The primary reason for caring about this is if you want to pair to another transmitter such as the Walkera with DeviationTx.  These are identifiable based on the color of the PCB when you open up the CX-10.  There is the Red, Green, and Blue.  There’s a lot of discussion on the Deviation TX board and RCGroups about which firmware is needed with which board (so go there for details), but in general I think that people have been able to make it work with all three versions.

cx-10 picts
Image from RCgroups.com forum post. 

IMG_1064I believe the Red and Green PCB versions came earlier. All of the units I purchased came from Amazon after Oct. 2015, and they have all been Blue PCBs.

The Cheerson CX-10A with Headless Mode

IMG_1118 Cheerson CX-10A Mini-Quadcopter with Headless Mode

Check price on Amazon. 

This is identical to the CX-10, but with an additional flying mode called “Headless Mode.” With the CX-10 and CX-10A in normal flying mode, the side with with the blue LEDs is the front (the “head”) of the quadcopter, and the side with the red LEDs is the back.  The forward/backward, and left/right control (by default on the right joystick of the controller) are all referenced to this “head” orientation to quadcopter.

In Headless mode, the head of the quadcopter stays fixed to where it was based on how it started at takeoff of the device.  This means that even if you rotate the the quadcopter with the left joystick and the blue/red LEDs indicate that the head of the quadcopter is rotating, it will not affect how the left/right controls work. When operating in headless mode, the LEDs will blink on and off (I guess they figure it’s ok for the LEDs to go off because they no londer indicate the heading).

This headless mode can make it less confusing for beginners to fly, but it doesn’t always work that great, so in general I never use it.  However, I have noticed that in headless mode it the yaw rate seems to be reduced, and that overall can make things it slightly easier to fly.

On Amazon, the CX-10A seems to cost about $2-$3 more, but not all the listings are accurate.  I’ve ordered a CX-10A at least once and received a normal CX-10. The packaging is the easiest way to tell the difference (see photos above), plus the symbol on the top middle of the body is different.

This CX-10A is also sold as the Cheerson Q4 where it is bundled with a handly little wrench to help pull off the propellers.

The Cheerson CX-10C with a Camera

Cheerson CX-10C Mini-Quadcopter with 0.3MMP Camera.

Check price on Amazon).  

 

The difference with this one is obvious, and it’s pretty amazing.  For an extra $10, you get a video camera that can record video or take still pictures to a micro SD card. The video below tells you everything you need to know.

The Cheerson CX-10D

Cheerson CX-10D Mini-Quadcopter with altitude hold.

Check price on Amazon).  

 

This model adds the altitude hold feature the CX-10A (there is no camera). This is a nice feature for beginners, because keeping the drone at an even altitude is one of the trickier things to learn at first.  Many folks bounce their drones off the ceiling and then the floor a few times before getting the hang of it.  The altitude control feature is implemented using a barometric sensor. There is a nice review of the CX-10D here.  

The Cheerson CX-10W with FPV

Cheerson CX-10W Mini-Quadcopter with Wifi FPV

Check price on Amazon).  

 

This version adds the amazing ability to stream images from the camera on the drone to your Android or iOS phone, which enables FPV (first-person view) flying.  It is fun to play with, but admittedly, using your phone to fly this is not the best interface.  Here is a thorough review of the CX-10W.

There is a third-party version of the app that enables you to fly via a CX-10C transmitter and then stream and record to your Android phone (the default app disables the transmitter capability once it is on).

The Cheerson CX-10WD with FPV and Altitude control

Cheerson CX-10WD Mini-Quadcopter with FPV and altitude hold.

Check price on Amazon).  

 

This version adds altitude capability to the CX-10W.  This unit is thoroughly reviewed as well.  

Fluoreon FX-10

The Floureon FX-10 is even smaller than the Cheerson CX-10.

 Check price on Amazon. 

If you don’t have enough room to fly the Cheerson CX-10 then you need a bigger living space, but maybe the Floureon FX-10 can help too. It includes headless mode, and flies about as well as the CX-10. Here is nice video review.

It is also sold as the Cheerson CX-Stars, the Mota JetJat Nano, or the DHD D1.

Mods and Hacks for the Cheerson CX-10

This is a compilation of links describing various hacks on the Cheerson CX-10 mini drone.

  1.  The first one is my own hack where I control a Cheerson CX-10 from a PC using a webcam.
    drone diagram
  2. This is a how-to on converting a Cheerson into a FPV.  Pretty amazing!

Cheerson CX-10 Smallest FPV

3. This post describes how to add shoulder buttons to control the yaw.  This is nice because with the Cheerson controller, when trying to rotate the yaw, the thrust is often inadvertently affected as well.

4. This forum post shows how a different RX can be hacked into the Cheerson.  In this case a DT RX32D was used.  This enabled custom firmware to be implemented.

5. Teardown and Video showing Cheerson controlled by Walker DeviationTX

6. Another Teardown.

7. This Gitub Gist has code for connecting an Arduino to the XN297 chip in the Cheerson TX. Enables one to control the Cheerson via the Arduino, or in this case, using the Arduino to read from another Flight TX into the Arduino.

Arduino Source code

View post on imgur.com

 

Dancing Penguin Robots!

View post on imgur.com

View post on imgur.com


Here’s directions on how to build a simple vibration robot using a battery holder, a vibration motor, and cable ties. I designed this to be a “maker” craft at my daughter’s penguin-themed birthday party, but I’m sure these can easily be adapted to whatever party theme you need.  Each of the party attendees helped with the final assembly and decorated these robots as penguins so they could participate in the “Penguin Robot Dance Party.”  My robot is the “Tux” robot (a.k.a. as the Linux mascot).

Materials and tools

 

vibration motor Vibrating motor

This is a DC motor with an offset weight stuck on the end.  They are used in massagers, game controllers, and other items that you want to vibrate.

battery holder AA Battery Holder with Switch

This battery holder has a switch, which is a must-have feature when you want to stop your robot.

Double-stick foam tape

You’ll need about 1 inch per robot.

Cable ties

You’ll only need 4 per robot, but buy some extras because they’re always useful to have around!

AA Battery

I used only 1 battery per robot, but you can use 2 per robot if you want your robot to move faster (see the discussion in the directions below).

Soldering iron

Obviously, only buy this if you don’t have one already.

Solder

You’ll need about an inch of this per robot.

Total cost for each robot was about $2.50 per robot (primarily for the motor, battery pack, and batteries).  The best place to get the vibration motor and battery pack is from Ebay. The other items are all on Amazon.  Click the links in the table above to see the prices.

Directions:

DSC_0346

Get your materials together.

DSC_0349

Solder the leads of the battery holder to the motor. I chose to use only one AA battery per robot.  It works fine with two batteries, but the vibration makes a louder sound, and it was not conducive to a festive party atmosphere.  Shorting out one side of the battery holder with a wire that I soldered allowed me to use just one battery and reduce the noise a bit.

DSC_0342

Use the double-stick foam tape to attach the motor to the battery pack. Attach the four cable ties so the tips point down and they look like legs.

Here is the file with the penguin robot that we let the kids color to decorate their penguins. After they colored them in, just tape it on.  Make sure the paper is not bumping against the motor to reduce the noise.

DSC_0350

You may need to fine-tune the position and angle of the cable ties.  Bending the cable tie of one leg toward the center of gravity tends to make the robot move more toward that leg.  But sometime it doesn’t, so just try a bunch of things and eventually it will work.

One interesting thing we discovered is that these robots can also be setup to follow along a wooden Thomas the tank engine train track.  You might need to tape the train tracks together with a piece of Scotch tape so there is no gaps, and adjust the width of the cable tie legs a bit.  You can either setup two parallel tracks and run “Penguin Races” or setup a loop, like shown below in the video.