Tuesday, July 31, 2018

Obstacles in the Water

Having eyes, see ye not? and having ears, hear ye not? and do ye not remember?
(Mark chapter 8 verse 18, King James Version)

Taking pictures or video with a camera is one thing, but interpreting meaning from those images is quite another. I attempted to read some literature on image recognition and obstacle detection, and got a few rough ideas from it, but nothing that really jumped out as a straightforward solution to the problem of how to detect obstacles floating around in the water, within the camera's field of view.

Since last week's approach of looking for color transitions to find the coast / horizon line seemed to work pretty well, I tried out a similar method of looking for large-ish deviations in color properties of 10x10 blocks of pixels to see whether or not a particular block of pixels might correspond to an object floating in the water. So far I have not tested out the approach on a wide variety of images, but I'm hopeful that by refining the algorithm it might prove to be useful. Below is a screenshot taken from a Windows program that I have been working on for collecting and viewing image statistics and testing out various object detection algorithms. The 10x10 blocks that corresponded to a floating object (i.e. my kayak) appear as small white squares:


We are going on a family vacation to Toronto this coming week, and although I am not permitted to bring AMOS with us in the van, I'm hopeful that I will still have some time to work on various object detection schemes whenever I get a bit of a break from the driving.

Tuesday, July 24, 2018

Red Lines on the Horizon

Not really too much to write about this week; I took AMOS back to Woolastook to capture some video footage:

https://youtu.be/1oR1ij-ENcQ


AMOS got caught in river grass a couple of times during the first part of this test. I'm going to have to figure out some way of shielding the propellers so that doesn't happen...

Later, I applied the feature detection (https://youtu.be/Q59GczzmsyY) and edge detection (https://www.youtube.com/watch?v=_Jexqwt0XSQ&feature=youtu.be) algorithms to the video frames to see how they worked for detecting features. As expected the features were quite susceptible to reflections and splashing in the water, but still might prove to be somewhat useful, perhaps if some form of filtering were used to remove features that don't actually correspond to obstacles in the water.

As a first step in obstacle detection, I worked on finding a way of figuring out where the horizon or coast line was in the various images. Then once that is found, other features can be considered important (i.e. obstacles) if they are below the horizon line.

Basically I looked for a color transition to indicate where the water ended and the trees / bush / sky began. I did this across the width of the image, and then found the linear line that matched up best with the found transition points. In most cases (I think > 90%, and close to 100% in open water) the horizon line was found correctly, even when there were other objects in the image, such as my kayak in the picture below, although the process used to find it was a bit computationally intensive, requiring anywhere between 0.2 to 2.5 seconds to find for each image.



The algorithm for finding the horizon / coast could still sometimes be fooled by reflections, or by large objects in the immediate foreground, including water droplets on the camera's Plexiglas window:




EDIT: Here is the video of the complete test with horizon lines added. Looking at the video, I can see that there are some issues when the waves are high and / or the sun is shining directly at the camera. Otherwise, in open water, the algorithm seems pretty good:

https://www.youtube.com/watch?v=P57pHC5y7lM&feature=youtu.be

Tuesday, July 17, 2018

Will it See in the Sea?

After adding the camera enclosure last week, I spent a number of days doing tests with bits of computer-vision code, trying to find something that might help AMOS with obstacle avoidance. To speed up the testing a bit, I modified the "BoatCaptain" software for the PC to request video capture frames from AMOS over our family WiFi network. It seemed to work pretty well, sending a new video frame from AMOS to my PC every 2 seconds, although it tended to slow down and get a bit laggy if AMOS was on the side of the pool furthest from the wireless extender.

So far I have experimented with two different types of feature detection. The first called the FAST algorithm (https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_fast/py_fast.html#) seemed to work reasonably well at finding corners in the environment, although it was also quite susceptible to reflections in the water, and I think might be difficult to use in practice. Here are some typical examples:


You can see in the above 2 examples that the number of features found is dependent on the "threshold" value (from 0 to 255) that is input to the algorithm. I found that a threshold value of somewhere between 60 and 70 was normally optimal for finding corners in the images. But of course not all corners are obstacles. Some correspond to objects in the far distance (i.e. the treetops). Others correspond to reflections in the water.

The second type of feature detection was called the Canny Edge Detector (https://docs.opencv.org/2.4/modules/imgproc/doc/feature_detection.html?highlight=canny#canny) and it worked pretty well for finding lines and edges, but would also find features that were just reflections in the water:



Of the two algorithms, the Canny Edge Detector seems like it would be a bit better suited to finding real obstacles. I don't know, maybe I'll need to use something else, probably non-vision based. Or maybe a hybrid approach. At any rate, sometime soon I'll take AMOS back out on the river to collect some video footage. The pool is OK for basic testing but it's a bit of a visual overload compared to what AMOS would typically see out on the open water.


Tuesday, July 10, 2018

pH and Pretty Pictures

I spent a couple of days working on some PC graphing software to draw sensor data directly on the GPS route maps that I have been creating so far. The result for the previous test at Cap-Brulé is pretty nice looking:


You can see that the water gets warmest where the stream empties into the ocean, which of course makes sense! Sometime when I get a chance (and the turbidity and pH sensors also online) it would be cool to do a grid pattern in this area.

I also hooked up the pH sensor and its circuit conditioning board (https://www.robotshop.com/ca/en/analog-ph-meter-pro.html) to AMOS, using an available +5V output to power it. The analog output of this thing is a bit more than my A to D chip can handle, which is running on 3.3 V power, and I accidentally fried the 4th channel of the A to D the first time I tried it out. Switching to channel 3 and using a resistor divider on the output solved the over-range issue. The documentation recommends calibrating it using known solutions and distilled water, but I just used tap water (assuming its pH to be 7) and vinegar (5% acetic acid by volume) which (according to Google) has a pH of about 2.4. It's probably OK for my prototyping purposes. I measured our pool water to have a pH of about 8, which might be a spec too high (again, according to Google).  I did try the sensor in our pool for an overnight test, but unfortunately something shutdown the program shortly after midnight. I couldn't find any record of a crash, and the Pi was still responsive and seemed to be behaving normally in the morning, so I'm not sure what happened there. I'm re-testing it in the pool tonight.

A while ago I had made a few attempts at 3D printing a waterproof enclosure for supporting one of the tiny little Raspberry Pi Camera boards. For various reasons all of these attempts failed, so I enlisted the help of the pros over at 3dhubs.com. They forwarded my 3D model to Rob's Print Service in Edmonton, and within minutes Rob was correcting my model (there were gaps in it and it definitely would not have been waterproof) and then a day or two later the part was finished. I just got it in the mail yesterday:

Today I threaded the PiCam ribbon cable through the neck and snapped the circuit board onto the mounting posts, and was relieved to see it snap onto the posts perfectly. I also made another large hole / potential leak source in AMOS and plopped the holder into place, tightening it with a large threaded nut that I printed earlier.


I also cut out a disk out of Plexiglas and screwed it onto the enclosure, with an o-ring sandwiched between.

Here are some pics that I took with the PiCam to test it out once it was in place:


Next step will be to try out some of the feature detection algorithms that I had previously tested out in the basement. Looking at the above pic, I guess reflections might be an issue in still water 😉




Wednesday, July 4, 2018

Coding & Testing

Some code was added to AMOS this week for logging general sensor data. At present, it is only logging the interior temperature of the boat and the water temperature (measured by the DS18B20 sensor added last week). The logging interval is controlled by editing a text configuration file.

Our family visited my parents in Shediac this weekend, which afforded a couple of opportunities for testing out AMOS in the ocean. The first test was disappointing, as it seemed as though AMOS had regressed; it was swerving back and forth, and had difficulty maintaining a straight line. Occasionally it would even turn in a complete circle before continuing on. I followed along in the kayak for about half an hour before I finally realized what must have happened: the software that had been added for measuring water temperature required ~ 0.75 seconds to acquire a single sample, and was happening in the same thread that collected data samples from the electronic compass. The compass output was at 10 Hz, but of course this got reduced to ~ 1 Hz with the new temperature stuff added in. Since the navigation routine now also ran at 10 Hz, and required up-to-date compass data (also at 10 Hz) in order to function correctly, it was no longer possible for AMOS to maintain a straight course.

So I changed the water temperature logging interval to once every minute, and a second test proved that it was much better:


My mother and sister also joined me in their kayaks for this test. Everything seemed to be working perfectly, but on the return trip west, we noticed that AMOS would repeatedly veer off in a southwesterly direction before correcting itself back northward on its return trip. After getting back, I noticed that one of the fastening nuts on the right propeller platform had come off, and the  right platform and propeller were both dangling from the U-bolt and swinging freely. So likely that had something to do with the meandering return trip. I'll need to remember to check those nuts for tightness prior to field testing... transporting the boat in our old van with its sketchy suspension system is probably not good for mechanical fastenings.

I also suspected that perhaps compass inaccuracy was to blame for the zig-zag course on the return trip. On Monday, I took the boat back to Woolastook and ran it through the following rectangular course:


There didn't seem to be any issues with maintaining a westerly course this time, so I guess the previous problem was just due to the nut coming off. The only deviation in this last course happened near the northeast corner as the boat was moving eastward. There was a very strong wind at the time, so I'm guessing it must have momentarily blown AMOS off-course.