Sufficient Air Power

Because AMOS is somewhat susceptible to getting stuck in stringy seaweed, I bought an inexpensive plastic air propeller (for use with remote control airplanes), and an electronic speed controller from Amazon. This past weekend some scrap wood was added to the back end of AMOS to mount it, and some simple test software was created to calibrate and use the speed controller. I really wasn't too sure how it was going to go, i.e. whether or not this little plastic propeller would produce enough thrust to move AMOS anywhere or not.

Here is a picture of  Checkers with AMOS and the new air propeller:

And a video of the propeller in action in our green pool:

https://www.youtube.com/watch?v=NDXu83YbiPo

It was extremely windy when the above video was shot, which likely accounted for some of the rotation that AMOS experienced. At present there are no rudders for directional control, I just wanted to see whether or not the propeller could produce enough thrust to move the boat. For the video, the thrust level was at about 70% of maximum output. This corresponded to a current draw of about 20 A at 12 V, comparable to the maximum amount of power that I used with the water propellers. The battery inside AMOS is rated for a maximum continuous output of 20 A, so I probably wouldn't want to drive it beyond that for any length of time. As expected, there is not as much propulsive thrust exerted by the air propeller, compared to the combined efforts of the two water propellers. However, now that the water propellers are removed, there is less drag in the water. I'm also wondering if a more hydrodynamic hull (perhaps catamaran style?) might also help to reduce drag.

The ribbon extension cable for the camera arrived last week, so that was hooked up and tested to confirm that it worked correctly. I also spent an hour or two searching around the Linux geek forums to figure out how to setup the Pi to automatically connect to any of a number of potential wireless access points. This sounds like it should be easy, I mean PCs have been doing that for years right? But there was actually a lot of conflicting information on the Internet about how to do it on a Raspberry Pi properly. I had tried and failed a few months earlier, and have always been manually editing the /etc/network/interfaces file and rebooting every time I wanted to switch access points. But this time I managed to get the wpa_supplicant.conf file setup properly, so now AMOS can connect to any of 3 potential access points, without the need for editing the configuration file every time.

The next step is going to be to design an air propeller cage (so I don't cut my fingers off) and a motor / rudder system mounted to the cage, behind the propeller for steering. I've been looking at lots of pictures of airboats online, and have some idea of how to do it, but I'm sure there will be some challenges. I also recently purchased a pair of wireless transceivers: https://www.robotshop.com/ca/en/24g-transceiver-rf220su-module.html that claim to have a line of sight range of up to 3 miles, and contain an Atmel AVR 8-bit microcontroller (the same one used in Arduino boards). One of the transceivers will be used on a small mast on AMOS, and besides maintaining a communications link, will be used to continuously monitor the "health" of the main Raspberry Pi board. If the Pi board becomes unresponsive for a set period of time, the smaller 8-bit micro can control a relay that will "hard-reboot" the Pi. The small micro also has pulse width modulated (PWM) outputs, analog to digital inputs, and digital inputs / outputs, but I haven't decided what (if anything) to do with them yet.

Airboat Revisited?

On Wednesday of last week I took AMOS out to Woolastook to try out the new navigation software, and see if perhaps it could collect a test sample grid of temperature and pH points in a 200 x 300 m area of the river. Unfortunately the new software for getting faster updates of the yaw angle (from the magnetometer) proved to be too noisy for the boat's simple turn function, so it would usually tend to fire the thrusters somewhat randomly trying to do a stationary turn. This function worked fine in the pool, but under real world conditions it was no good. To compound difficulties, I had also keyed in the GPS coordinates for the corners of the sampling grid incorrectly, so that the boat was always attempting to drive itself back to shore.

That night, the software was modified to use a filtered average yaw angle from the magnetometer, based on the previous second's worth of data. This restored the original functionality of the simple turn function. The coordinates of the sample grid were also fixed, and the following day, the test was re-attempted. Things started off well and AMOS proceeded into the wind towards the northwest corner of the sample grid. It then proceeded eastward to complete the first row of sampling, but started behaving quite erratically as can be seen in the GPS track shown below:

After about an hour of this, it mysteriously died. I think that possibly the software crashed, although there was no record in the ship's log file or any of the Raspberry Pi log files to indicate what might have caused the crash. The ship's log file indicated that the battery level was OK before the crash, so presumably power was not an issue. One possibility is that some wires might have shorted out inside AMOS, causing the Pi to stop running. There is some reason to suspect this, since it was discovered after the fact that the compass module had pulled out of its enclosure and was free to float around inside the boat. It was this floating compass that was responsible for the erratic navigation during the eastward sampling stage. The next day, Velcro was applied to the compass module to hold it in place within its enclosure, and a second log file was created for the shell software that starts the main AMOS program, to hopefully save the exit code if a crash occurs.

The next couple of days (Thanksgiving) were spent at Mom and Dad's cottage, wiring up the new A to D board that arrived from ControlEverything.com. It was pretty close to what I had before in terms of parts, just professionally soldered and put together, so I'm sure it will be more reliable. There were some minor software changes to make for it too... some of the configuration commands were slightly different, and I needed to do some averaging in software to achieve an 18-bit level of precision. So far it seems to be working quite well. The weather didn't cooperate for most of the weekend, but on Monday, the waves were not too bad, so I released AMOS for what was planned to be a short 1 km test. Unfortunately AMOS only went about 75 m north from the shore, and then struggled to move anywhere... eventually the wind (which was coming from the north) pushed it back to shore where I retrieved it and discovered that the thrusters had both become fouled with seaweed, the leftmost one worse than the right. Below are a picture of the thruster with the seaweed in it and the clump of seaweed by itself after I picked it out by hand:

So apparently the new propeller shield is not very effective, at least not for this common variety of seaweed. So today I spent some time googling various types of airboat designs. This one (https://www.cim.mcgill.ca/~mrl/pubs/anqixu/iros2011_boat.pdf) seemed intriguing, and indicated that a vessel of a size similar to AMOS could be driven using air propellers. I ordered some inexpensive drone equipment from Amazon (https://www.amazon.ca/dp/B07C5KYNY7/ref=pe_3034960_236394800_TE_dp_1) so once that arrives I'll attach it to AMOS and see whether or not it can actually push the boat through the water. 

Tidying Up The Engine Room

I’m tired of sailing my little boat
Far inside the harbor bar --
I want to go out where the big ships float
Out on the deep where the great ones are.
And should my frail craft prove too slight
For waves that sweep those billows o'er,
I'd rather go down in the stirring fight
Than drowse to death by the sheltered shore.

-- author unknown


Two weeks ago I posted a picture of the inside of AMOS after it had been driven home in the van from a test and some of the components had shifted around a bit:

Although the above picture exaggerated the usual condition of AMOS's innards, it wasn't too far from the usual reality. It had gotten to the point where I seemed to be making repairs to the wiring as often as not, and it was time to straighten things up a bit in order to save time going forward.. 

The enclosures that were designed the week before arrived from 3DHubs, so all of the wiring was painstakingly untangled, de-soldered in some cases, the components placed into their respective enclosures, and then re-wired and re-soldered. By some minor miracle, all the electronics worked as they did before the operation took place. A master wiring technician might scoff at the result, but I think that it is a huge improvement.

Without the cover plate:

And with the cover plate:

A ribbon extension cable is on order for the camera, so that it can properly reach from the outside of the boat to the interior of the box.

I had originally intended to build my own printed circuit board to replace the semi-reliable ADC board (the yellowish brown prototyping board in the top-left corner of the above 2 photos). And one morning I set out to do that, but quickly realized that it was going to take weeks. Surely there was something on the Internet already put together? There actually didn't seem to be very much that was suitable, but I did manage to find this: https://store.ncd.io/product/4-channel-i2c-0-20v-analog-to-digital-converter-with-i2c-interface/ It's only 16-bit vs. the 18-bit that I have now, but its sampling rate is 4 times faster, so by maintaining a running average of 4 samples in the software, I should get the equivalent of 18-bit precision. It also accepts a wider input voltage than what I have now, which should help to simplify some of the signal conditioning electronics.

Lastly, some good improvements were made in AMOS's software for holding a fixed heading while driving forward smoothly. The previous control software was usually able to make the boat move forward at a fixed heading, but it was not smooth, requiring the rapid on / off firing of the left and right thrusters to maintain direction, while sacrificing forward speed. The new control algorithm is smoother and faster, and still keeps the boat following a forward course through the water. It has only been tried in the pool so far, but I hope to have it out in the river soon.

A Week of Repairs

Although I had hoped to get AMOS out at least once this past week for more testing, there were a number of repair-related tasks, mostly to do with the turbidity sensor that I wanted to take care of first. I also modified the grid collection software, as described last week, to alternate from East to West, then West to East as it collects points in a grid pattern, and cleaned up some of the other sampling code to get rid of occasional extra unwanted samples that were being collected.

The week before I had filled the turbidity sensor enclosure (read cat-food tin) with epoxy in a futile attempt to make it waterproof. Perhaps a different brand of epoxy, one that you mix together would have worked better. The kind I used required no mixing and was meant to cure when exposed to air, which meant that the bulk of the epoxy within the enclosure remained in liquid form, and then leaked quickly out under small cracks of the enclosure whenever it was tilted. The epoxy also made its way in through small openings in the top of the plastic turbidity sensor enclosure and covered the transmitter, receiver, and various other electrical components. So this week, I slowly picked away bits of epoxy from the turbidity sensor, but eventually discovered that the pins on the connector had weakened and snapped off anyway due to corrosion caused by exposure to water.

So I ordered a replacement cheap-o turbidity sensor and hope to have it in a week or two. It appears to be based on a design that is commonly used in washing machines. I'll have to try to cover it as best I can (duct tape?), and just hope that it doesn't get too wet. I was thinking that maybe by attaching some foam floats to the sides of the "sensor boat" it will lift it up a bit out of the water, and hopefully keep its top a little bit more dry. I also looked into some real industrial quality turbidity sensors and got a few quotes. The prices for decent models that are completely waterproof and capable of operating in sea water (titanium construction) are between $2K and $4K CAD. They also include "wipers" for performing self-cleaning of the optical windows. At the moment (although I'm tempted) there doesn't seem to be much point in buying one of these, as both the cheap $14 model and the $2K+ models have similar analog outputs that effectively look the same as far as AMOS and its software are concerned. Of course the $14 model is not waterproof, nor is it calibrated or guaranteed to be stable. It is also (unless externally shielded) affected by external light (i.e. the sun). But it will do for now.

I tried to replicate an issue with the software that occurred in the test from the week before, in which the software stopped writing to the ship's log after beginning the grid data sampling routine. I ran a number of tests in the pool to try to replicate the same sampling conditions, but was unable to replicate the problem: saving to the ship's log seemed to work fine no matter what.

I also bought a small sheet of rubber from Amazon and experimented with making my own gasket for use with the cheap turbidity sensor and various household containers. This was an abysmal failure. Even when I thought I had something that looked fairly tight, water leaked in immediately. Luckily I didn't waste too much time on it. Any waterproofing on AMOS will need to be bought from others.

While testing the turbidity sensor, I noticed that the A to D board was also glitching in and out. My wiring on this thing is admittedly pretty bad, but I think the root cause of the glitching might have  been a number of damp (blue) desiccant beads that had adhered to the surface and pins of the nearby compass module board. Both the compass module and the A to D board share the same I2C bus, so it's conceivable that communication problems could result from an electrical fault on one or both of these boards. On two separate occasions, cleaning off desiccant from the pins of the compass module brought the A to D board back to life. At no point though was the compass module ever unresponsive, so the desiccant theory is really just a guess.

As a step in the right direction towards better protecting the circuit boards on AMOS, I designed some enclosures and sent them off to 3DHubs for printing. Here is the enclosure that will house the main Raspberry Pi Compute module, the GPS module, the leak sensor, and the two thruster speed controllers:

The enclosures are not waterproof, but rather are intended to organize the boards and wires and keep things from shorting out. They also have lids to help protect against any water that might splash down from above.

Fog Vision (or Lack Thereof)

Friday morning was a bright clear day at home, but there was a thick fog hovering over the river at Woolastook. The original plan was to test out both the grid sampling software and the LiDAR vision safety software, but it was immediately apparent that the LiDAR would not work in the fog. Back scatter of light from the water droplets must have resulted in the "false" object distance of around
1.8 m. This required turning off the LiDAR safety flag in the configuration file for AMOS, in order to allow the software to ignore the fog and drive the boat normally. Ordinarily, I guess it is OK that AMOS would not drive in the fog, but I'm wondering if the same problem would also happen with rain, or maybe snow? I suspect it would, which could be a bit of an issue with this particular sensor I guess. Reading briefly on the subject of LiDAR issues with various types of weather, it seems that some hardware is better than others in this respect.

The boat proceeded towards the northeast corner of a 9 x 11 point grid that it was supposed to sample, although it did not quite make it. The software crashed, and the boat veered off to the northeast. When it crashed, the exit code was actually zero, so the software did not automatically restart as it was supposed to. I'll need to find some other way of detecting that a crash occurred I guess. Even if the software did restart though, it would have started back at the original GPS coordinate, which would have taken too long, given that only the morning was available for the test. I have since added a command line option to the program to allow it to continue at the last known step of the previous program instance. Looking at the ship's log later, it seems that the crash was likely related to some code for correcting magnetometer headings using GPS data. The code did not appear to be working correctly anyway, so for now at least it has been removed.

Recent improvements to logging code for the server have shown that numerous entities from the Internet seem to enjoy trying to connect and send data to the server. Typically they do not stay connected for a long length of time, and so far have not caused any noticeable harm, but potentially they could tie up the server, or even issue a random command to the boat that causes it to crash or become unresponsive. To remedy that, I have added a simple password requirement to the server, so that both the boat and the boat captain (i.e. mobile app) need to send this password within 10 seconds after connecting, or otherwise be disconnected. This seemed to work when tested today, as it bounced numerous unwanted connections all day long.

A repeat of the grid test was attempted again today. This time the weather was clear, so it was possible to test the LiDAR safety function. I maneuvered my kayak in front of the boat's path several times, and each time the software would stop the thrusters until I moved out of view. Success!!!

This time the software did not crash, and the boat had time to take one "line" of samples before it became necessary to head back. The next version of AMOS will need to be much more hydrodynamic. It took about 80 minutes for it to drive itself almost 2 km to the sampling grid; and was quite difficult to pull back, as I only allotted 30 minutes for the return trip 😓.

While idling along in the kayak behind AMOS, I realized that the grid sampling software needs to be improved: currently it always uses the same east-west direction for each of the sampling lines. However, it would be more efficient to alternate the east-west direction for each line. Getting out in the boat is always the best way to find improvements. Luckily our weather has been good and the water is still warm, but probably there are only a few weeks of good weather left, so I'll have to squeeze in as many tests as possible!

(NOTE: I believe the curvy blue path in the eastern part of the above image is due to the thruster wires sometimes getting jostled so that they are in close proximity to the magnetometer. Current flowing through the wires interferes with the magnetometer, which in turn causes the software to detect an invalid heading and fire the thrusters erratically, which in turn causes more heading errors, etc. The result is a characteristic, jerky, weaving path. The same thing also happened during some pool testing this past weekend. Some time soon I want to fix up the jumble of wires comprising the innards of this boat.)

Not Quite Invincible

To test out how well (or not) the propeller shields worked as a protection against weeds, the following fairly grungy location at Woolastook was selected for a field trial:

As might be expected, this amount of vegetation was too much. The grass and other plant material plastered themselves against the intake grate, effectively stopping the boat in its tracks. At least the grass did not get caught in the propeller though. Here is a picture of a large clump of grass that hung from the propeller shield after the boat had been dragged through the section pictured above:

AMOS was dragged out beyond the grassy area, and then piloted using the Android app for a while, but after about 10 minutes it died abruptly. Checking the ship log afterward showed that the battery was not well-charged at the start of the test (it had been collecting turbidity data all night long the night before), and manually driving the boat near top speed was enough to make the battery drop momentarily below 10 volts, causing the battery management system to kick in and force a reboot of the Pi computer.

Testing over the past few weeks, it was noticed that the GPS signal would sometimes drop out for periods of time. Unfortunately the antenna cable had stopped working (probably the cable had a break in it somewhere) and it was no longer providing any gain so it was replaced with a new $17 model from Amazon. Testing with the gpsmon program on the Pi showed that the new one provided about 3 or 4 dB of gain.

This past week the Android app has gotten a couple of new functions: "homing" and "picture snapshot":

Homing brings AMOS back to the GPS location of the phone and the picture snapshot function tells AMOS to snap a picture using the camera and send it to the phone.

A second test was performed at a different location in Woolastook to test out the LiDAR. The camera was programmed to snap a picture anytime the LiDAR detected an obstacle within 13 m. There were some initial communication problems during this test, which happened to result in a dramatic crash against some rocks:

https://youtu.be/1HZaNorxaYQ

Luckily nothing was damaged, and after pulling the boat away from the rocks, the test continued. The rocks and my kayak were the only things that the LiDAR detected during this test. The water was quite calm and placid though, so the test should be repeated sometime when there are some actual waves, to see if LiDAR reflection off of waves might be an issue. After this test, some code was added to automatically shut down the thrusters whenever an object is detected within 13 m distance.

Dirty Pool Turbidity Testing

An unfortunate chemical imbalance in our pool this past long weekend resulted in some algal growth and clouding of the water. Normally this would be cause for disappointment, given the warm weather we have been having lately. However it was a good opportunity to test out the cat-food-tin turbidity sensor. The following is the turbidity voltage data that has been collected since about 4:00 pm this afternoon:

The good news is that the chemical re-balancing that Kelly performed earlier today appears to be working. This was the state of the pool at the start of data collection (~ 4:00 pm):

At the time of this writing it is dark outside, but I'll post a picture of the (hopefully) cleared pool tomorrow morning. Other turbidity readings collected over the past week were usually around 3.6 or 3.7 V when the pool was clear. At Cap-Brulé this past weekend, the turbidity voltage was between 3.35 V and 3.5 V. That was for a moving boat however, which could possibly have some air bubbles interfering with the readings a bit.

Some new software for AMOS was also tested at Cap-Brulé. It changed how the log file was saved, in order to open/close the file for each write, ensuring no log data gets lost in the event of a crash. Also, a secondary program was created to launch the main AMOS program, and then re-launch it if necessary in the event of a crash. Since implementing these changes, no crashes have been observed, but if a crash does happen, AMOS will be ready!

Here is a picture of Hannah holding one of the new 3D printed propeller shields:

These shields are quite light and only contribute a modest amount of drag. They will hopefully protect the propellers from the worst types of entangling seaweed. The water at Cap-Brulé was relatively weed-free this weekend, so a more harsh test will be done tomorrow at Woolastook in an area that has a lot of lily pads.

EDIT: At 07:20 this morning (Sept. 05) the pool is now much more clear. The turbidity sensor is  averaging ~ 3.7 V: