b4064 - web tag behaviour

[freddie]

I've just tried upgrading to b4064 but I'm having an issue with web tags. I use web tags in a number of database queries. The one that is giving me a problem is the UserTemp1 web tag. This one holds a 30 cm depth temperature. Just before I upgraded (and unknown to me) the depth temperature sensor stopped sending data to my GW1100 gateway. This normally manifests itself as MX using 0.0 in the web tag value. However, on upgrading, it is now sending '-' (hyphen). I thought it would just be an empty value if there was no data, as nulls are now being stored internally to MX (rather than using 0.0 placeholder). My database query can cope with an empty value as it accepts null. But it can't cope with a hyphen as it is not a number or null.

[Ulric2]

There are going to be all sorts of hiccups associated with the change from zero to null for "missing" data in various fields and I suspect this is one of them.

In databases, there is an explicit NULL value which can be coalesced to zero or "-" on output if it is appropriate. In the text dayfiles and logs, the previous default zero seems to be replaced by an empty string. These changes can have odd effects anywhere these values are consumed, including webtags, output reports, graphs and third party uploads. In turn, any user written code which consumes these values may throw errors because a hyphen, as you correctly point out, is not a number. In general, the data type of the output needs to be appropriate to the destination.

[HansR]

Nobody said using null values would come without reprogramming effort at places. If you use your own software you may have to do something yes.

In the logfiles - I think that is what @Ulric2 is talking about? - there are no empty strings. The null value is registered as nothing (two subsequent comma's). That means that is you go to the original data you will have to reprogram and act on two subsequent comma's.

I assume in the SQL databases - values created by CMX - it is a true null value so when reading a record you must read it with a proper variable (accepting null values).

But in user written code you have to take action yourself.
And yes, reporting and charts in CMX no doubt may have effects but that is for CMX to solve.

For webtags it seems that the destination - human reader or something automated - gets a defining role here for what output it should give.
I am not sure that is what we want. Neither am I sure that would be necessary.

[mcrossley]

I've long thought about handling null values in web tags.

C1 and MX both output a dash (or multiple dashes) for web tags that have no value. This is because of the original intent of web tags - to be used for embedding text in HTML files.

Increasingly web tags are being used to generate code, and the string dash(ed) output is less than optimal.

My two thoughts were:

1. Have a global setting to switch web tag "null" value output to be a specified string. You could then define an empty string, "null" or whatever you want.
2. Add an parameter for all tags that lets you specify a per web tag value for null.
3. Have both.

Both options come their own issues. With the global setting, existing code will probably break all over the place, and need to be amended. Setting per tag is pretty messy for users.

[HansR]

Given that not all webtags require a special handling - e.g. only few will act as input for queries - I would suggest (two pennies):

1. Have a global setting for a default output for a null value which default currently is - and can remain - a dash, users may change this in (2)
2. Supply a non-compulsory setting per webtag that overrules the global setting e.g. <#temp rc=y nv=null>
3. Possible values for nv (null value) can be null, "defined string" or numeric value.

Note: It is a consideration to not have the global setting but simply use "-" as it can be easily changed by the nv parameter.

[mcrossley]

I have implemented a per web tag nv=xxx parameter for the next build.

[freddie]

This is great, thank you

[HansR]

Second that