Red Lines on the Horizon

Not really too much to write about this week; I took AMOS back to Woolastook to capture some video footage:

https://youtu.be/1oR1ij-ENcQ


AMOS got caught in river grass a couple of times during the first part of this test. I'm going to have to figure out some way of shielding the propellers so that doesn't happen...

Later, I applied the feature detection (https://youtu.be/Q59GczzmsyY) and edge detection (https://www.youtube.com/watch?v=_Jexqwt0XSQ&feature=youtu.be) algorithms to the video frames to see how they worked for detecting features. As expected the features were quite susceptible to reflections and splashing in the water, but still might prove to be somewhat useful, perhaps if some form of filtering were used to remove features that don't actually correspond to obstacles in the water.

As a first step in obstacle detection, I worked on finding a way of figuring out where the horizon or coast line was in the various images. Then once that is found, other features can be considered important (i.e. obstacles) if they are below the horizon line.

Basically I looked for a color transition to indicate where the water ended and the trees / bush / sky began. I did this across the width of the image, and then found the linear line that matched up best with the found transition points. In most cases (I think > 90%, and close to 100% in open water) the horizon line was found correctly, even when there were other objects in the image, such as my kayak in the picture below, although the process used to find it was a bit computationally intensive, requiring anywhere between 0.2 to 2.5 seconds to find for each image.

The algorithm for finding the horizon / coast could still sometimes be fooled by reflections, or by large objects in the immediate foreground, including water droplets on the camera's Plexiglas window:

EDIT: Here is the video of the complete test with horizon lines added. Looking at the video, I can see that there are some issues when the waves are high and / or the sun is shining directly at the camera. Otherwise, in open water, the algorithm seems pretty good:

https://www.youtube.com/watch?v=P57pHC5y7lM&feature=youtu.be

Will it See in the Sea?

After adding the camera enclosure last week, I spent a number of days doing tests with bits of computer-vision code, trying to find something that might help AMOS with obstacle avoidance. To speed up the testing a bit, I modified the "BoatCaptain" software for the PC to request video capture frames from AMOS over our family WiFi network. It seemed to work pretty well, sending a new video frame from AMOS to my PC every 2 seconds, although it tended to slow down and get a bit laggy if AMOS was on the side of the pool furthest from the wireless extender.

So far I have experimented with two different types of feature detection. The first called the FAST algorithm (https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_fast/py_fast.html#) seemed to work reasonably well at finding corners in the environment, although it was also quite susceptible to reflections in the water, and I think might be difficult to use in practice. Here are some typical examples:

You can see in the above 2 examples that the number of features found is dependent on the "threshold" value (from 0 to 255) that is input to the algorithm. I found that a threshold value of somewhere between 60 and 70 was normally optimal for finding corners in the images. But of course not all corners are obstacles. Some correspond to objects in the far distance (i.e. the treetops). Others correspond to reflections in the water.

The second type of feature detection was called the Canny Edge Detector (https://docs.opencv.org/2.4/modules/imgproc/doc/feature_detection.html?highlight=canny#canny) and it worked pretty well for finding lines and edges, but would also find features that were just reflections in the water:

Of the two algorithms, the Canny Edge Detector seems like it would be a bit better suited to finding real obstacles. I don't know, maybe I'll need to use something else, probably non-vision based. Or maybe a hybrid approach. At any rate, sometime soon I'll take AMOS back out on the river to collect some video footage. The pool is OK for basic testing but it's a bit of a visual overload compared to what AMOS would typically see out on the open water.

pH and Pretty Pictures

I spent a couple of days working on some PC graphing software to draw sensor data directly on the GPS route maps that I have been creating so far. The result for the previous test at Cap-Brulé is pretty nice looking:

You can see that the water gets warmest where the stream empties into the ocean, which of course makes sense! Sometime when I get a chance (and the turbidity and pH sensors also online) it would be cool to do a grid pattern in this area.

I also hooked up the pH sensor and its circuit conditioning board (https://www.robotshop.com/ca/en/analog-ph-meter-pro.html) to AMOS, using an available +5V output to power it. The analog output of this thing is a bit more than my A to D chip can handle, which is running on 3.3 V power, and I accidentally fried the 4th channel of the A to D the first time I tried it out. Switching to channel 3 and using a resistor divider on the output solved the over-range issue. The documentation recommends calibrating it using known solutions and distilled water, but I just used tap water (assuming its pH to be 7) and vinegar (5% acetic acid by volume) which (according to Google) has a pH of about 2.4. It's probably OK for my prototyping purposes. I measured our pool water to have a pH of about 8, which might be a spec too high (again, according to Google).  I did try the sensor in our pool for an overnight test, but unfortunately something shutdown the program shortly after midnight. I couldn't find any record of a crash, and the Pi was still responsive and seemed to be behaving normally in the morning, so I'm not sure what happened there. I'm re-testing it in the pool tonight.

A while ago I had made a few attempts at 3D printing a waterproof enclosure for supporting one of the tiny little Raspberry Pi Camera boards. For various reasons all of these attempts failed, so I enlisted the help of the pros over at 3dhubs.com. They forwarded my 3D model to Rob's Print Service in Edmonton, and within minutes Rob was correcting my model (there were gaps in it and it definitely would not have been waterproof) and then a day or two later the part was finished. I just got it in the mail yesterday:

Today I threaded the PiCam ribbon cable through the neck and snapped the circuit board onto the mounting posts, and was relieved to see it snap onto the posts perfectly. I also made another large hole / potential leak source in AMOS and plopped the holder into place, tightening it with a large threaded nut that I printed earlier.

I also cut out a disk out of Plexiglas and screwed it onto the enclosure, with an o-ring sandwiched between.

Here are some pics that I took with the PiCam to test it out once it was in place:

Next step will be to try out some of the feature detection algorithms that I had previously tested out in the basement. Looking at the above pic, I guess reflections might be an issue in still water 😉

Coding & Testing

Some code was added to AMOS this week for logging general sensor data. At present, it is only logging the interior temperature of the boat and the water temperature (measured by the DS18B20 sensor added last week). The logging interval is controlled by editing a text configuration file.

Our family visited my parents in Shediac this weekend, which afforded a couple of opportunities for testing out AMOS in the ocean. The first test was disappointing, as it seemed as though AMOS had regressed; it was swerving back and forth, and had difficulty maintaining a straight line. Occasionally it would even turn in a complete circle before continuing on. I followed along in the kayak for about half an hour before I finally realized what must have happened: the software that had been added for measuring water temperature required ~ 0.75 seconds to acquire a single sample, and was happening in the same thread that collected data samples from the electronic compass. The compass output was at 10 Hz, but of course this got reduced to ~ 1 Hz with the new temperature stuff added in. Since the navigation routine now also ran at 10 Hz, and required up-to-date compass data (also at 10 Hz) in order to function correctly, it was no longer possible for AMOS to maintain a straight course.

So I changed the water temperature logging interval to once every minute, and a second test proved that it was much better:

My mother and sister also joined me in their kayaks for this test. Everything seemed to be working perfectly, but on the return trip west, we noticed that AMOS would repeatedly veer off in a southwesterly direction before correcting itself back northward on its return trip. After getting back, I noticed that one of the fastening nuts on the right propeller platform had come off, and the  right platform and propeller were both dangling from the U-bolt and swinging freely. So likely that had something to do with the meandering return trip. I'll need to remember to check those nuts for tightness prior to field testing... transporting the boat in our old van with its sketchy suspension system is probably not good for mechanical fastenings.

I also suspected that perhaps compass inaccuracy was to blame for the zig-zag course on the return trip. On Monday, I took the boat back to Woolastook and ran it through the following rectangular course:

There didn't seem to be any issues with maintaining a westerly course this time, so I guess the previous problem was just due to the nut coming off. The only deviation in this last course happened near the northeast corner as the boat was moving eastward. There was a very strong wind at the time, so I'm guessing it must have momentarily blown AMOS off-course.

Battery is Back and AMOS Meets the Public

Good news: the lithium battery just needed a bit of a voltage boost from a plug in battery charger. The folks at Ukko Versatile Energy Solutions (https://www.ukko.io/) offered me some helpful advice: basically the battery management system (BMS) inside the lithium battery pack has a safety switch to cutoff output when the voltage falls below 10 volts. But because the solar charger on AMOS requires some battery voltage in order to turn on, it wouldn't even start to charge the battery, no matter how brightly the sun was shining.

So I bought a $20 wall charger from Amazon and used it for 3 hours to bring the battery back to about 13 volts. To help avoid this problem in the future, AMOS now has some improved software to avoid getting too close to the dangerous 10 volt cutoff. If the measured voltage drops below 12 volts, it only allows half-speed operation, and if it drops below 11 volts, it shuts down the thrusters altogether. Once the voltage gets back to 13 volts, everything goes back to normal. Now that I think about it, it should probably also send an alert email / text to notify the user about the low voltage condition. I should probably also do some tests to see exactly how long the battery can last if it is close to being depleted. It's a bit tricky to know exactly how depleted the battery is, since its voltage tends to shoot back up whenever the thrusters are turned off.

AMOS went through a couple more field trials this week: the first was in the usual spot, but because it was a Saturday, my daughters Kirsten and Bexie joined me and we brought our dog Checkers and the canoe. AMOS started out well for about a minute, but then appeared to stop in the middle of the cove and start moving in circles. We paddled back to shore with AMOS in tow and I set about clearing some lake grass from the propellers and checking over the source code while the girls and Checkers paddled around the cove. I made a few minor changes to the software, but nothing really seemed to work: the boat would always just move a little ways, and then start moving in a circle, not always in the same direction. There was a lot of grass getting stuck in the propellers, and it seemed that that was the likely culprit; a more complete check of the ship logs later that evening and a more thorough cleaning of the propellers confirmed it.

The second trial happened the next day, just a little further down the road at a small boat launch. This time I went by myself, but a family of 4 was pulling up in a motorboat just as I was getting AMOS into the water. A young boy, probably about 12 was curious:

"Is that a robot boat?"

"Well, yes it is actually!"

"Cool! I like anything to do with technology... does it use an Arduino or a Raspberry Pi?"

"It uses a Raspberry Pi, and some other electronics, The Raspberry Pi handles most of the navigation, motor control, and stuff."

"I was going to learn Python, but I figured it would probably take FOUR hours or something, so I haven't done it yet."

"I use C++ myself, although I think most people now prefer to use Python. You might find that it takes a little bit more than four hours to learn though..."

The boy would have liked to see AMOS in action, but I was afraid of it getting stuck in the grass again, so this time I towed it out through the shallows and started it on the other side of a bridge in much deeper water. The following is the (nice and straight) track that AMOS followed:

It stopped mysteriously before reaching its final destination though, and I wasn't sure at the time what had happened. A quick check over the Bluetooth link with my phone confirmed that the program was not running, so I assumed that it had crashed. Fearing that some water had gotten in again (despite adding some layers of duct tape around the problematic U-bolts) I turned off the thruster power switch, hitched AMOS to the kayak and started back. While in the process of doing this, a man and woman driving a large pontoon boat came over to inquire about what I was doing:

"Are you alright there?"

"Oh yes, I'm fine thanks!"

"Did you find that thing in the water?"

"Ummm, no it's mine actually, I'm just testing it."

"Do you need us to take it somewhere for you?"

"No, that's OK, I think it just crashed. It happens a lot actually."

They seemed a bit perplexed about this, but then motored off somewhere else, leaving me to tow it back to the boat launch. Inside the hull, there was no evidence of water, and none of the leak sensors had detected anything. Looking at the ship log on shore confirmed that AMOS had not in fact crashed, but its program had been stopped by a keyboard press, presumably sent somehow by accident over the phone's Bluetooth link. The phone had been stored in the kayak's dry storage container, so I'm not exactly sure how this might have happened. It was quite windy with some waves though, so perhaps some jostling in the container was enough to momentarily turn on or register "screen touches" that were sent to the boat. I'll have to remember to turn off the terminal program for the next test. In order to more quickly diagnose these sorts of stoppages, I wrote a small program yesterday that just prints out the last 10 lines of text from the most recent ship's log file. Something that I should be able to access and read quickly while sitting in a kayak in the middle of a river.

Today I replaced the lithium battery back in AMOS, and also added a water-proof temperature sensor, the DS18B20. The wiring changes weren't much, but took me almost 3 hours, as I'm terrible at soldering and need to check, double-check, and triple-check the schematics and board layout to make sure I'm not going to wreck anything. Luckily, nothing got wrecked, and the temperature sensor appears to be working. I plan to just tow it along behind AMOS, a few inches below the surface of the water.

Death in the Night

The propulsion and course correction software for AMOS went through several iterations this week, all in a grand effort to make the boat drive in a straight line. The first couple of iterations actually made things worse; I didn't even need to use my kayak for those as AMOS either veered off towards the bank to the west or traveled about 30 m and then spun crazily around in a circle. Last night, I brought the laptop with me, intent on coding and re-coding until things improved.

After a couple of tweaks, things did improve and AMOS drove through the cove in a mostly straight line. I think it covered this distance (~ 500 m) the fastest it had ever done. Once the boat left the cove though, the wind was much stronger from the west and the water became quite choppy. Although the boat was pitching and rolling quite a bit, it seemed to move with a sense of purpose through the water. Whenever it approached one of the GPS destinations though, it would often veer off, or double back. I'm guessing that perhaps it was moving too fast near the destination, but I'll need to have a look at the ship's logs later on to confirm this. At one point a freezie wrapper became lodged between the blades of the left propeller creating a loud noise and causing the boat to spin around in circles.
I removed the wrapper, but about 10 minutes later, AMOS mysteriously stopped meandering around and just stayed put. It was after 8:30 by this point, so I turned off its power switch, hitched on the tow rope, and pulled it back to shore. Looking inside the hatch I could see that the leak sensor LED was lit, so although no water was visible, some drops must have gotten in; not surprising I guess given the choppy conditions and the imperfect sealing that has been applied over various holes made to the top of the boat.The computer was still responsive and behaving normally though.

That night, I left the boat in the garage to dry out, but still plugged in to the battery since everything still seemed to be running normally, even though I guess a few drops of water had gotten in.

This morning I awoke to find the interior dead! Normally there are a host of red, green, and blue LEDs merrily blinking away inside the hull, but this morning all was dark. A quick test of the Lithium battery showed that its voltage was zero. Nada. Zip. Nothing. So now I have to figure out what to do. I'm not sure if the battery can be saved or not, or what might have happened to it. I'm wondering if perhaps last night's test might have pulled its charge down too much? And then maybe just operating the computer overnight was enough to fully deplete it? Or maybe the little bit of water that got in somehow caused problems? If it can be saved I'll need to find a wall charger I guess, since the solar charger that I use requires the battery to have enough charge to power the charger.

(UPDATE: AMOS is now back online thanks to a $70 sealed lead acid battery from Canadian Tire. It is significantly larger and heavier than the Lithium one though, so hopefully the manufacturer will respond to the email that I sent with some suggestions for how to bring it back. The ship's log and other operating system logs seem to indicate that a reboot occurred at the end of last night's test. I'm not sure exactly why the reboot occurred; possibly it was a result of the water intrusion. Just a few minutes ago I squeezed the little sponge on one of the leak sensors near a U-bolt that supports the left propeller, and a few drops of water came out. That's the only water I found inside though, and the only leak sensor that indicated a leak.)

Woolastook Navigation Challenge Parts II, III, and IV

One of the things I got around to adding this past week was a nice little on / off motorcycle switch with a blue "on" light for controlling power to the thrusters' (+12 V power supply):

I also added a check in the main software for AMOS to look for a high to low transition in this switched voltage and this tells the program to close and the Raspberry Pi to reboot itself. This makes it easier for me to re-establish a Bluetooth link to the boat later. The switch itself makes recovering from mishaps a lot better, as I no longer have to wrestle with the hatch or fumble with my phone while trying to avoid falling out of the kayak.

Three attempts were made this week to complete the Woolastook challenge (the same route from last week). The first test had some issues in and around the cove with following the wrong heading... basically if the heading was off by more than about 10 degrees it would crash into the shore. I did a sort of crude 8-point compass calibration after that, and then re-tested, with the following result:

This second test was pretty good on the trip out, but crashed into the shore a number of times on the way back. Some of the crashes actually looked pretty hard and bent one of the solar panel rails a bit, but AMOS kept functioning and was able to extract itself from the shore each time, until near the end when it got securely stuck in some overhanging branches near the shore. The blue path of the boat shows how it would sometimes set out at an invalid heading, and then gradually correct its heading as it got closer to the waypoint. This observation gave me the idea to try splitting up the course (in AMOS' navigation software) into 10 interpolated stages between the main waypoints. The result (the 4th and finally successful) Woolastook challenge is shown below:

There was still one close call on this last test where AMOS nearly crashed into the shore, and there were some obvious problems with consistently maintaining a desired direction, especially when going into or perpendicular to the wind, which was quite strong for this test. I made some improvements today to the software used for trimming the propeller speeds though, so I think that will help a lot to maintain a more straight course. It looked promising in the pool, and I will get back to the river in a couple of days probably to try it out.