My first IOT thing

iHally · 21 August 2017 05:12

Esp8266 and other web enabled chips and their boards have been around for a while .

It was time I used the ones I’d bought. Having scored a few days off work, I dusted off the parts box and started bread boarding. but what to make! There are so many things you can make with a low power web enabled device but i thought it was best to revive an old project.

A long time ago I tired measuring the depth of water using the BMP085 which has been discontinued and superseded by several chips the newest being the BMPE280 which also measures humidity.

That project had a huge flaw, when the atmospheric pressure changed during a measurement the depth measurement would be incorrect by as much as half a meter :-/

Now that I’m living on tank water, accurate water depth is important in my life. Also useful for aquariums etc. I must note there are plenty of other ways to measure water depth/height including sonar on water surface, light reflections, float switches, conductive squish tube (cant remember the name). However they all suffer from environmental influences or accuracy problems.

This setup uses 2 bmp180’s on an i2c bus and toggling a reset register on each sensor to select between them for readings. This way the pressure in the tank and of the atmosphere can be measured to increase accuracy.

They are hooked up to a Weemos D1 mini clone programmed via an Arduino IDE. I’ll link to Gitub when the code isn’t in such a terrible state.
Here is a snippet of code:

#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>
#include "HTTPSRedirect.h"
#include <Wire.h>
#include <Adafruit_BMP085.h>

HTTPSRedirect allows for direct posting to a google sheet for data storage via a google script. Without this you are doomed to submit to the cost and limitations of one of the online services like Thingspeak, Adafruit.io, data.sparkfun, thinger.io…

If I was actually to deploy this on my tanks it is important to have a few things worked out. Especially handing Wifi outage and the Power system.

For the WiFi the ESP8266WiFiMulti library was used. This allows you to code in several access points. Very useful for me since I have 5 routers handling the internet on my property. But you can include your phone hotspot so you can get data in case the home router goes out or starts disliking your iot thing for any reason.

For the power hacking together a reclaimed lipo some cheap solar light solar cells and a charge board made for a nice long term low power system. However this setup only supplies 20mA (0.02A) in full sun which we can assume will only happen for 4-5 hours a day. So this means…

We need to sleep. Putting the esp8266 to sleep reduces its power consumption considerably. From about 70-150mA (0.07A idle - 0.15A during wifi comms) to 20uA (0.00002A).

However in my setup with a voltage divider measuring the battery and the two pressure sensors I had a 630uA (0.00063A) draw during deep sleep. So on a 3000mA battery you could power sleep for over 200 days or run with WiFi on for almost one day. However if the ESP8266 is only awake for 10 seconds every 10 minutes and asleep the rest other the time, it should only need a battery (0.416mah awake 0.695mAh asleep) totaling 1.035mAh for an hour or 24.852 mAh for a day. Now on a 3000mA battery the setup would last 120 days.

Lets include our measly solar charging. 20mAh for 4.5 hours a day could charge a flat 90mAh battery each day and we only need 24.852mAh. This means we have surplus power to account for miss-aligned solar cells, my bad math, cloudy days and for times the ESP8266 is awake for longer than it should be. Worse case scenario the sensor setup will need to be charged 3 times a year. I’d call this deployable.

Now I just need to make the housing so I can attach it to my tank and the get the graphing of the data working properly.

Hamish_McGregor · 22 August 2017 01:05

Amazing project, but why use pressure? Using a float or ultrasonic range finder will allow a device to be mounted in the top of the tank and give an accurate level assuming you know the distance to the bottom.

iHally · 22 August 2017 10:05

Thanks @Hamish_McGregor. Ultrasonic sensors are so versitile and fortunately i’ve had a bit of experiance with them. A few commercial products are available for just this use case, however none of them are less than $10 or ‘web enabled’.

They would be cheap enough to make but suffer from a few issues.

dead band - this is to prevent unwanted reflection immediately infront of the sensor giving false distance readings. 2) unwanted reflections - water is often not flat (citation needed) and tank walls can mess with the reading too. 3) deployment - I would need to cut a hole in the top centre of my tanks to use it. Thats something I’m not able to do on all of my tanks.
Which is why I opted for my approach which was cheap, non invasive and revived an old and unfinished project

iHally · 4 September 2017 09:09

Whilst postponing the construction a housing I have been working on the back end and discovered that Javascript is not my forte.

If anyone can point out the obvious there that would be great

function getMaxInColumn(sheet, column) {
  var colArray = sheet.getRange(2, column, sheet.getLastRow()).getValues();
  var maxInColumn = colArray.sort(function(a,b){return b-a})[0][0];
  return maxInColumn;
}

function getMinInColumn(sheet, column) {
  try{
    var colArray = sheet.getRange(2, column, sheet.getLastRow()).getValues();
    //colArray = colArray.filter(Number);
    var colArray = colArray.filter(function(val){return Boolean(val)});
    var minInColumn = colArray.sort()[0][0]; //colArray.sort(function(a,b){return a-b})[0][0]; //colArray.sort()[0][0];
  }
  catch(error){
    Logger.log(JSON.stringify(error));
    return error; //"";
  }
  return minInColumn;
}

The Max function works flawlessly however type errors or null outputs are returned for the MIN function. You can see i’ve had a few attempts but I need some assistance to figure this one out.

This is part of a Google Apps Script that is exicuted every time the esp8266 passes values through a html address. I thought it would be interesting to collect max and min values of the weather over the period of a year. I didnt realise filtering null, NAN, “”, none vales from an array would be so challenging. I fear the only thing missing is to import a library for this functionality or more likely something dreadfully obvious.

Howver I do have the graphing working acceptably here

Thanks for your help.

Redhatter · 4 September 2017 09:37

I’ll admit, I’m no guru where it comes to JavaScript… I know enough to be dangerous. Sometimes I find tools like filter and map more difficult to understand, and I’m not sure about the performance implications of sorting everything then just picking the first (or last) element.

I presume sheet.getRange(…).getValues(); just returns an array of numbers. Based on this, I’d be doing the following:

/**
 * Retrieve the maximum value seen in a column.
 */
function getMaxInColumn(sheet, column) {
    var colArray = sheet.getRange(2, column, sheet.getLastRow()).getValues();
    var maxSeen = undefined;
    var cell;
    for (cell in colArray) {
        if ((maxSeen == undefined) || (maxSeen < cell))
            maxSeen = cell;
    }
    return maxSeen;
}

/**
 * Retrieve the minimum value seen in a column.
 */
function getMinInColumn(sheet, column) {
    var colArray = sheet.getRange(2, column, sheet.getLastRow()).getValues();
    var minSeen = undefined;
    var cell;
    for (cell in colArray) {
        if ((minSeen == undefined) || (minSeen > cell))
            minSeen = cell;
    }
    return minSeen;
}

That avoids having to sort the array first, thus should be able to run in a single pass.

Redhatter · 4 September 2017 09:41

In fact, just thinking about that, one cause might be it’s trying to do comparisons/arithmetic with non-numeric data… so you might also want to throw into the top of the loop (both functions):

    if (typeof cell != 'number')
        continue;

devians · 4 September 2017 09:45

https://jsfiddle.net/xwfobszq/1/

your filter comparitor isnt ideal, see the one in the fiddle

theres a second example there where you premap it through Number, just depends on how dirty your data is.

devians · 4 September 2017 09:46

Also, doing stuff like this is probably what you actually want: https://stackoverflow.com/questions/1669190/find-the-min-max-element-of-an-array-in-javascript

Redhatter · 4 September 2017 09:51

That StackOverflow response looks on the money to me. Should be a lot quicker since you’re using a built-in library.

iHally · 6 September 2017 11:13

Thanks @devians and @Redhatter for your help! I’ll give those a try.

I really appreciate you taking the time to provide links and type out the actual code i needed!!!

iHally · 9 September 2017 22:58

Finally I have been gifted a morning with the baby asleep to play with this.

I tried the exampls you both provided, I’m not sure if it is just how I implemented it but for the life of me I could not get the to work. I kept getting undefined or empty values.

I finally played around with the code trying different values to see how the sort() function actually works.
It seems sort without function returns the array or values sorted from lowest to highest. However i could not seem to address the last value in the array without error. There is something weird going on with the google apps script.

This code works:

function getMaxInColumn(sheet, column) {
var colArray = sheet.getRange(2, column, sheet.getLastRow()).getValues();
var maxInColumn = colArray.sort(function(a,b){return b-a})[0][0];
return maxInColumn;
}

function getMinInColumn(sheet, column) {
var colArray = sheet.getRange(2, column, sheet.getLastRow()).getValues();
var maxInColumn = colArray.sort()[1];
return maxInColumn;
}

Part of the problem could be that values are passed from the esp8266 as strings and entered into the sheet as such. This would explain the errors and weird results.
For the moment I’m happy it works. Time will have to be set aside later on in the project to learn undunderstand the apps script preferences and javascript itself.

Thanks again for the help!

Redhatter · 9 September 2017 23:28

Okay, so if you have strings in those cells, you may need to convert it all to numeric data before attempting to find the min/max values.

There’s a handy array method, map, that iterates over an array, calling a function you supply to it with each value present in that array. So for example:

[1,2,3,4,5].map(function (val) { return val + 1; });

would return

[2,3,4,5,6]

Using that polyfill mentioned in @devians’ post (put this somewhere in the top of your script):

Array.prototype.max = function() {
  return Math.max.apply(null, this);
};

Array.prototype.min = function() {
  return Math.min.apply(null, this);
};

… you should be able to get what you’re after with this:

function getMaxInColumn(sheet, column) {
    return sheet.getRange(2, column, sheet.getLastRow()).getValues().map(function (v) {
        return parseInt(v);
    }).max();
}

That should:

retrieve the column (getRange call)
Taking the value returned by getRange, call its getValues method
With the value returned by getValues, calls the map method with a function that converts each string element to a number… map should return an array of numbers
With the return value from map, we call the max method that was added earlier (the Array.prototype.max function above)

The minimum function should work similar:

function getMinInColumn(sheet, column) {
    return sheet.getRange(2, column, sheet.getLastRow()).getValues().map(function (v) {
        return parseInt(v);
    }).min();
}

electronicsguy · 4 December 2017 17:04

Hello all. Version 2 of HTTPSRedirect is posted on Github: HTTPSRedirect. Please use only this version henceforward. It comes with many new features and corrected errors. Please post your comments and questions on Github.