Tuesday, September 25, 2018

A Week of Repairs

Although I had hoped to get AMOS out at least once this past week for more testing, there were a number of repair-related tasks, mostly to do with the turbidity sensor that I wanted to take care of first. I also modified the grid collection software, as described last week, to alternate from East to West, then West to East as it collects points in a grid pattern, and cleaned up some of the other sampling code to get rid of occasional extra unwanted samples that were being collected.

The week before I had filled the turbidity sensor enclosure (read cat-food tin) with epoxy in a futile attempt to make it waterproof. Perhaps a different brand of epoxy, one that you mix together would have worked better. The kind I used required no mixing and was meant to cure when exposed to air, which meant that the bulk of the epoxy within the enclosure remained in liquid form, and then leaked quickly out under small cracks of the enclosure whenever it was tilted. The epoxy also made its way in through small openings in the top of the plastic turbidity sensor enclosure and covered the transmitter, receiver, and various other electrical components. So this week, I slowly picked away bits of epoxy from the turbidity sensor, but eventually discovered that the pins on the connector had weakened and snapped off anyway due to corrosion caused by exposure to water.

So I ordered a replacement cheap-o turbidity sensor and hope to have it in a week or two. It appears to be based on a design that is commonly used in washing machines. I'll have to try to cover it as best I can (duct tape?), and just hope that it doesn't get too wet. I was thinking that maybe by attaching some foam floats to the sides of the "sensor boat" it will lift it up a bit out of the water, and hopefully keep its top a little bit more dry. I also looked into some real industrial quality turbidity sensors and got a few quotes. The prices for decent models that are completely waterproof and capable of operating in sea water (titanium construction) are between $2K and $4K CAD. They also include "wipers" for performing self-cleaning of the optical windows. At the moment (although I'm tempted) there doesn't seem to be much point in buying one of these, as both the cheap $14 model and the $2K+ models have similar analog outputs that effectively look the same as far as AMOS and its software are concerned. Of course the $14 model is not waterproof, nor is it calibrated or guaranteed to be stable. It is also (unless externally shielded) affected by external light (i.e. the sun). But it will do for now.

I tried to replicate an issue with the software that occurred in the test from the week before, in which the software stopped writing to the ship's log after beginning the grid data sampling routine. I ran a number of tests in the pool to try to replicate the same sampling conditions, but was unable to replicate the problem: saving to the ship's log seemed to work fine no matter what.

I also bought a small sheet of rubber from Amazon and experimented with making my own gasket for use with the cheap turbidity sensor and various household containers. This was an abysmal failure. Even when I thought I had something that looked fairly tight, water leaked in immediately. Luckily I didn't waste too much time on it. Any waterproofing on AMOS will need to be bought from others.

While testing the turbidity sensor, I noticed that the A to D board was also glitching in and out. My wiring on this thing is admittedly pretty bad, but I think the root cause of the glitching might have  been a number of damp (blue) desiccant beads that had adhered to the surface and pins of the nearby compass module board. Both the compass module and the A to D board share the same I2C bus, so it's conceivable that communication problems could result from an electrical fault on one or both of these boards. On two separate occasions, cleaning off desiccant from the pins of the compass module brought the A to D board back to life. At no point though was the compass module ever unresponsive, so the desiccant theory is really just a guess.

As a step in the right direction towards better protecting the circuit boards on AMOS, I designed some enclosures and sent them off to 3DHubs for printing. Here is the enclosure that will house the main Raspberry Pi Compute module, the GPS module, the leak sensor, and the two thruster speed controllers:

The enclosures are not waterproof, but rather are intended to organize the boards and wires and keep things from shorting out. They also have lids to help protect against any water that might splash down from above.

Tuesday, September 18, 2018

Fog Vision (or Lack Thereof)

Friday morning was a bright clear day at home, but there was a thick fog hovering over the river at Woolastook. The original plan was to test out both the grid sampling software and the LiDAR vision safety software, but it was immediately apparent that the LiDAR would not work in the fog. Back scatter of light from the water droplets must have resulted in the "false" object distance of around
1.8 m. This required turning off the LiDAR safety flag in the configuration file for AMOS, in order to allow the software to ignore the fog and drive the boat normally. Ordinarily, I guess it is OK that AMOS would not drive in the fog, but I'm wondering if the same problem would also happen with rain, or maybe snow? I suspect it would, which could be a bit of an issue with this particular sensor I guess. Reading briefly on the subject of LiDAR issues with various types of weather, it seems that some hardware is better than others in this respect.

The boat proceeded towards the northeast corner of a 9 x 11 point grid that it was supposed to sample, although it did not quite make it. The software crashed, and the boat veered off to the northeast. When it crashed, the exit code was actually zero, so the software did not automatically restart as it was supposed to. I'll need to find some other way of detecting that a crash occurred I guess. Even if the software did restart though, it would have started back at the original GPS coordinate, which would have taken too long, given that only the morning was available for the test. I have since added a command line option to the program to allow it to continue at the last known step of the previous program instance. Looking at the ship's log later, it seems that the crash was likely related to some code for correcting magnetometer headings using GPS data. The code did not appear to be working correctly anyway, so for now at least it has been removed.

Recent improvements to logging code for the server have shown that numerous entities from the Internet seem to enjoy trying to connect and send data to the server. Typically they do not stay connected for a long length of time, and so far have not caused any noticeable harm, but potentially they could tie up the server, or even issue a random command to the boat that causes it to crash or become unresponsive. To remedy that, I have added a simple password requirement to the server, so that both the boat and the boat captain (i.e. mobile app) need to send this password within 10 seconds after connecting, or otherwise be disconnected. This seemed to work when tested today, as it bounced numerous unwanted connections all day long.

A repeat of the grid test was attempted again today. This time the weather was clear, so it was possible to test the LiDAR safety function. I maneuvered my kayak in front of the boat's path several times, and each time the software would stop the thrusters until I moved out of view. Success!!!

This time the software did not crash, and the boat had time to take one "line" of samples before it became necessary to head back. The next version of AMOS will need to be much more hydrodynamic. It took about 80 minutes for it to drive itself almost 2 km to the sampling grid; and was quite difficult to pull back, as I only allotted 30 minutes for the return trip 😓.

While idling along in the kayak behind AMOS, I realized that the grid sampling software needs to be improved: currently it always uses the same east-west direction for each of the sampling lines. However, it would be more efficient to alternate the east-west direction for each line. Getting out in the boat is always the best way to find improvements. Luckily our weather has been good and the water is still warm, but probably there are only a few weeks of good weather left, so I'll have to squeeze in as many tests as possible!

(NOTE: I believe the curvy blue path in the eastern part of the above image is due to the thruster wires sometimes getting jostled so that they are in close proximity to the magnetometer. Current flowing through the wires interferes with the magnetometer, which in turn causes the software to detect an invalid heading and fire the thrusters erratically, which in turn causes more heading errors, etc. The result is a characteristic, jerky, weaving path. The same thing also happened during some pool testing this past weekend. Some time soon I want to fix up the jumble of wires comprising the innards of this boat.)

Tuesday, September 11, 2018

Not Quite Invincible

To test out how well (or not) the propeller shields worked as a protection against weeds, the following fairly grungy location at Woolastook was selected for a field trial:

As might be expected, this amount of vegetation was too much. The grass and other plant material plastered themselves against the intake grate, effectively stopping the boat in its tracks. At least the grass did not get caught in the propeller though. Here is a picture of a large clump of grass that hung from the propeller shield after the boat had been dragged through the section pictured above:

AMOS was dragged out beyond the grassy area, and then piloted using the Android app for a while, but after about 10 minutes it died abruptly. Checking the ship log afterward showed that the battery was not well-charged at the start of the test (it had been collecting turbidity data all night long the night before), and manually driving the boat near top speed was enough to make the battery drop momentarily below 10 volts, causing the battery management system to kick in and force a reboot of the Pi computer.

Testing over the past few weeks, it was noticed that the GPS signal would sometimes drop out for periods of time. Unfortunately the antenna cable had stopped working (probably the cable had a break in it somewhere) and it was no longer providing any gain so it was replaced with a new $17 model from Amazon. Testing with the gpsmon program on the Pi showed that the new one provided about 3 or 4 dB of gain.

This past week the Android app has gotten a couple of new functions: "homing" and "picture snapshot":

Homing brings AMOS back to the GPS location of the phone and the picture snapshot function tells AMOS to snap a picture using the camera and send it to the phone.

A second test was performed at a different location in Woolastook to test out the LiDAR. The camera was programmed to snap a picture anytime the LiDAR detected an obstacle within 13 m. There were some initial communication problems during this test, which happened to result in a dramatic crash against some rocks:


Luckily nothing was damaged, and after pulling the boat away from the rocks, the test continued. The rocks and my kayak were the only things that the LiDAR detected during this test. The water was quite calm and placid though, so the test should be repeated sometime when there are some actual waves, to see if LiDAR reflection off of waves might be an issue. After this test, some code was added to automatically shut down the thrusters whenever an object is detected within 13 m distance.

Tuesday, September 4, 2018

Dirty Pool Turbidity Testing

An unfortunate chemical imbalance in our pool this past long weekend resulted in some algal growth and clouding of the water. Normally this would be cause for disappointment, given the warm weather we have been having lately. However it was a good opportunity to test out the cat-food-tin turbidity sensor. The following is the turbidity voltage data that has been collected since about 4:00 pm this afternoon:

The good news is that the chemical re-balancing that Kelly performed earlier today appears to be working. This was the state of the pool at the start of data collection (~ 4:00 pm):

At the time of this writing it is dark outside, but I'll post a picture of the (hopefully) cleared pool tomorrow morning. Other turbidity readings collected over the past week were usually around 3.6 or 3.7 V when the pool was clear. At Cap-Brulé this past weekend, the turbidity voltage was between 3.35 V and 3.5 V. That was for a moving boat however, which could possibly have some air bubbles interfering with the readings a bit.

Some new software for AMOS was also tested at Cap-Brulé. It changed how the log file was saved, in order to open/close the file for each write, ensuring no log data gets lost in the event of a crash. Also, a secondary program was created to launch the main AMOS program, and then re-launch it if necessary in the event of a crash. Since implementing these changes, no crashes have been observed, but if a crash does happen, AMOS will be ready!

Here is a picture of Hannah holding one of the new 3D printed propeller shields:

These shields are quite light and only contribute a modest amount of drag. They will hopefully protect the propellers from the worst types of entangling seaweed. The water at Cap-Brulé was relatively weed-free this weekend, so a more harsh test will be done tomorrow at Woolastook in an area that has a lot of lily pads.

EDIT: At 07:20 this morning (Sept. 05) the pool is now much more clear. The turbidity sensor is  averaging ~ 3.7 V: