Serveral parsing issues for DSMR v3.0

[marcelrv]

I may have an older unit, as seems the lib does not properly parse my messages

see here what my meter outputs:

/KMP5 ZABF000000000000
0-0:96.1.1(205C4D246333034353537383234323121)
1-0:1.8.1(00185.110*kWh)
1-0:1.8.2(00084.020*kWh)
1-0:2.8.1(00013.030*kWh)
1-0:2.8.2(00019.040*kWh)
0-0:96.14.0(0001)
1-0:1.7.0(0000.98*kW)
1-0:2.7.0(0000.03*kW)
0-0:17.0.0(999*A)
0-0:96.3.10(1)
0-0:96.13.1()
0-0:96.13.0()
0-1:24.1.0(3)
0-1:96.1.0(3238313031453631373038389930337131)
0-1:24.3.0(120517020000)(08)(60)(1)(0-1:24.2.1)(m3)
(00124.477)
0-1:24.4.0(1)
!

The most obvious missing from my meter is the CRC...
The other parsing issues:

0-0:17.0.0(999*A)
               ^
Invalid unit

and

(00124.477)
^
OBIS id Empty

I worked around the CRC issue, but the gasmeter value on separate line seems more tricky

[marcelrv]

I worked around the OBIS id Empty by hacking parse.h. At least I can parse the message.
Though the gasmeter value can't be read normal

 // Parse data lines
    while (line_end < end) {
      if (*line_end == '\r' || *line_end == '\n') {
        if (!(*(line_end+1) =='(' || *(line_end + 2) =='(')){
        ParseResult<void> tmp = parse_line(data, line_start, line_end, unknown_error);
        if (tmp.err)
          return tmp;
        line_start = line_end + 1;
      }
      }
      line_end++;
    }

To get the gasmeter I added a field like this and some manual parsing
DEFINE_FIELD(gas_deliveredS, String, ObisId(0, GAS_MBUS_ID, 24, 3, 0), RawField);

[matthijskooijman]

Thanks for your report and examples. I would be happy to get DSMR v3.0 supported correctly. It's been a while since I worked on this, so I had to read up a bit on DSMR again. I noticed that the spec links in my README were broken, so updated the README and added a bunch of spec documents to this repo.

CRC
It seems that the DSMR v3.0 spec does not specify a checksum, just a ! to conclude the message. This is probably best fixed with an option somewhere, probably in the parser (which means either a runtime argument to the P1Parser::parse function, or a template argument to the P1Parser class (where the latter might make more sense). The argument could either be a DSMR version number (deriving whether a checksum is needed from that, a simple boolean enable_crc or something, or an object/struct containing an enable_crc which allows more options later). Any thoughts on that?

Invalid unit for 0-0:17.0.0
The DSMR v3 spec seems to specify the "A" unit for this value. DSMR v4 has removed this value. The current code has kW, which is also in the parse example, so I suspect that's what my DMSR v4 meter produced. I don't think multiple units for a single value are supported currently, but it's probably better to follow the spec (and perhaps recommend people not to parse this field, since it means "The actual threshold Electricity in A" and might not actually be meaningful (it's 999 for both our meters...)).

Gas meter 0-1:24.3.0 format / obis id empty
There's two things happening here. First, the entry is wrapped over two lines. In the specs, I can't find anything definitive about how different entries are separated (it is implied that there' is one entry per line, but the example in the v3.0 spec shows exactly the same gas reading spread over 2 lines, as well as a long text message spread over multiple lines). The v4 and v5 specs say "Every line (except the last one) is ended with a CR/LF (Carriage Return / Line Feed).", but still does not say how different entries map to lines (though the example has longer entries on one a single line as well).

To support linebreaks at arbitrary places we can do some heuristics about whether a line is really completed (as you're doing now), or perhaps the proper way to parse this is to not decide where one line/entry ends in advance, but simply parse an entry and allow newlines at any point. This does require some structural changes, but I think it should be possible.

Second, the format used for the entry is weird. The v3 specification specifies F, but the value shown in your message and the v3 spec example seems to match the "Profile generic" message, which can contain one or more measurements (each with their own id and unit). In both your case and the example, a single 0-1:24.2.1 measurement is contained, which is not specified in the v3.0 specification. In the v4.0 specification, this 0-1:24.2.1 message is specified and only used as a top-level message.

Supporting a nested format like this is probably possible, where during the parsing of one field, another field is parsed, but this might also require some significant changes.

It's a bit of a pity that these DSMR specs are such a mess, which leaves us guessing...

I might not have time soon to pick up these things, but I'm happy to hear your thoughts (if I haven't gone way over your head already with implementation details :-p).

[marcelrv]

Thanks for the detailed answer.
I saw (after I coded my hack) there is already a PR for making CRC optional #18

Invalid unit for 0-0:17.0.0
This indeed is little useful value. Would not waste much time on it.