DLMS Push listener on Raspberry pi

By hallmar, 7 May, 2021

Forums

Hello,

I am having problems with the Push Listener Python example on a Raspberry Pi.
I get the messages with correct framing; messages start and end with 0x7E but the script just spits out the raw values and translates nothing. I would think that it would spit out the DLMS codes in translated .xml format for the received bytes, like here https://www.youtube.com/watch?v=OcmYvcsgjz4, but it only shows the raw bytes.

I will start with noting what kind of equipment I have right now:
Raspberry Pi 2
M-Bus to Serial TTL converter

This is how one received frame looks like:

New data is received. /dev/ttyAMA0:7E A8 87 CE FF 03 13 86
New data is received. /dev/ttyAMA0:B7 0F 00 00 A3 57 0C 07 E4 08 0D 04 16 13 0F FF
New data is received. /dev/ttyAMA0:80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00
New data is received. /dev/ttyAMA0:00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01
New data is received. /dev/ttyAMA0:09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00
New data is received. /dev/ttyAMA0:00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00
New data is received. /dev/ttyAMA0:01 08 00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 07 00 FF 6F 5E 7E

When I put this through the XML translator(after parsing the data myself) I get this:

1: 7E A8 87 CE FF 03 13 86 B7 0F 00 00 A3 57 0C 07 E4 08 0D 04 16 13 0F FF 80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00 00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01 09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00 00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 07 00 FF 6F 5E 7E
<HDLC len="86" >

<TargetAddress Value="33FF" />
<SourceAddress Value="1" />

<FrameType Value="13" />
<NextFrame Value="0F0000A3570C07E4080D0416130FFF800000020E010E020412002809060008190900FF0F02120000020412002809060008190900FF0F01120000020412000109060000600100FF0F02120000020412000809060000010000FF0F02120000020412000309060100010800FF0F02120000020412000309060100010700FF" />
</HDLC>

I then convert the Value = " " using PDU converter:

And with this I used the OBIS helper to look up the information in human language that the DLMS frame was giving me. Although I do get three errors at the end...

Are all of these extra steps neccessary or is there a script available to do this?

Thank you for your help!

Hi,

Data of the push message is not fit to one frame. You need to read all frames before you can parse the received data. Have you try with this?

https://github.com/Gurux/Gurux.DLMS.Python/tree/master/Gurux.DLMS.Push…

Try with parameters like:
python main.py -S /dev/ttyAMA0 -t Verbose

BR,
Mikko

Hi Mikko,

Yes this is the script that I ran.
I tried with the Verbose parameter but I didn't really get anything out of it.

This is an example:

New data is received. /dev/ttyAMA0:00 80 00 00 00 80 80 00 00 00 78 00 78 E0 00 80 00 00 00 00 78 E0 00 78 E0 00 00 F8 78 FE 00 78 00 80 80 00 00 00 80 00 00 00 80 80 00 00
trace:08:19:48 2 78 00 78 E0 00 80 00 00 00 00 80 78 00 00 F8 78 FE 00 80 00 80 80 00 00 00 80 00 00 00 80 80 00 00 00 00 78 E0 00 80 00 00
New data is received. /dev/ttyAMA0:78 00 78 E0 00 80 00 00 00 00 80 78 00 00 F8 78 FE 00 80 00 80 80 00 00 00 80 00 00 00 80 80 00 00 00 00 78 E0 00 80 00 00
trace:08:19:48 2 00 78 00 00 00 F8 78 FE 00 80 00 80 80 00 00 00 80 00 00 00 80 80 00 00 F8 00 78 E0 00 80 00 78 00 00 78 00 00 00 00
New data is received. /dev/ttyAMA0:00 78 00 00 00 F8 78 FE 00 80 00 80 80 00 00 00 80 00 00 00 80 80 00 00 F8 00 78 E0 00 80 00 78 00 00 78 00 00 00 00
trace:08:19:48 2 F8 78 FE 00 80 00 80 80 00 00 00 80 00 00 00 80 80 00 00 F8 00 78 E0 00 80 00 78 00 00 78 00 F8 00 00 F8 78 00 F8 78 1E 00 00 80 80
New data is received. /dev/ttyAMA0:F8 78 FE 00 80 00 80 80 00 00 00 80 00 00 00 80 80 00 00 F8 00 78 E0 00 80 00 78 00 00 78 00 F8 00 00 F8 78 00 F8 78 1E 00 00 80 80

Hi Mikko,

One other thing: The meter that I'm reading from sends out a push message every 5 seconds. Here is a link to a video where I showcase this: https://youtu.be/hMCNwkSrmeA

I would think that it sends one whole frame every 5 seconds yes? Unless I'm hugely misunderstanding the Push message concept.

all the best,
Hallmar

Hi,

Your meter is configured to send push messages. That allows you to get current data, but you can't make the connection to the meter. This is done to improve security. You can get current readings, but you can't establish the connection to the meter.

It seems that your serial port settings are wrong. Received data is not valid. Check the baud rate first, you can try 9600.
When your serial port settings are correct the received data should start and end for 7E.

BR,
Mikko

Hi Mikko,

I accidentally set the baud rate to 4800 but it should have been 2400 like I used in the beginning.
Here is the string with the correct Baud rate with the Verbose parameter:

gurux_dlms version: 1.0.104
gurux_net version: 1.0.17
gurux_serial version: 1.0.15
/dev/ttyAMA0:2400 8EVEN1
Press any key to close the application.
Media state changed. MediaState.OPENING
Media state changed. MediaState.OPEN

trace:09:00:19 2 7E A8 87 CE FF 03 13 86
New data is received. /dev/ttyAMA0:7E A8 87 CE FF 03 13 86
trace:09:00:19 2 B7 0F 00 01 6E 7A 0C 07 E4 08 10 07 10 22 01 FF
New data is received. /dev/ttyAMA0:B7 0F 00 01 6E 7A 0C 07 E4 08 10 07 10 22 01 FF
trace:09:00:19 2 80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00
New data is received. /dev/ttyAMA0:80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00
trace:09:00:19 2 00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01
New data is received. /dev/ttyAMA0:00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01
trace:09:00:19 2 09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00
New data is received. /dev/ttyAMA0:09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00
trace:09:00:19 2 00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00
New data is received. /dev/ttyAMA0:00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00
trace:09:00:19 2 00 02 04 12 00 03 09 06 01 00 01 07 00 FF 61 71 7E
New data is received. /dev/ttyAMA0:00 02 04 12 00 03 09 06 01 00 01 07 00 FF 61 71 7E

trace:09:00:23 2 7E A8 87 CE FF 03 13 86
New data is received. /dev/ttyAMA0:7E A8 87 CE FF 03 13 86
trace:09:00:23 2 B7 0F 00 01 6E 7B 0C 07 E4 08 10 07 10 22 05 FF
New data is received. /dev/ttyAMA0:B7 0F 00 01 6E 7B 0C 07 E4 08 10 07 10 22 05 FF
trace:09:00:23 2 80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00
New data is received. /dev/ttyAMA0:80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00
trace:09:00:23 2 00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01
New data is received. /dev/ttyAMA0:00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01
trace:09:00:24 2 09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00
New data is received. /dev/ttyAMA0:09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00
trace:09:00:24 2 00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00
New data is received. /dev/ttyAMA0:00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00
trace:09:00:24 2 00 02 04 12 00 03 09 06 01 00 01 07 00 FF 93 02 7E

New data is received. /dev/ttyAMA0:00 02 04 12 00 03 09 06 01 00 01 07 00 FF 93 02 7E
trace:09:00:28 2 7E A8 87 CE FF 03 13 86
New data is received. /dev/ttyAMA0:7E A8 87 CE FF 03 13 86
trace:09:00:28 2 B7 0F 00 01 6E 7C 0C 07 E4 08 10 07 10 22 0A FF
New data is received. /dev/ttyAMA0:B7 0F 00 01 6E 7C 0C 07 E4 08 10 07 10 22 0A FF
trace:09:00:28 2 80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00
New data is received. /dev/ttyAMA0:80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00
trace:09:00:29 2 00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01
New data is received. /dev/ttyAMA0:00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01
trace:09:00:29 2 09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00
New data is received. /dev/ttyAMA0:09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00
trace:09:00:29 2 00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00
New data is received. /dev/ttyAMA0:00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00
trace:09:00:29 2 00 02 04 12 00 03 09 06 01 00 01 07 00 FF 6E B6 7E
New data is received. /dev/ttyAMA0:00 02 04 12 00 03 09 06 01 00 01 07 00 FF 6E B6 7E

Image

Hi,

What is the meter manufacturer and model? I believe that the meter is expecting ACK from the client before sending the next frame. This is new functionality, that has added to DLMS recently and I haven't seen any meter that is implementing it. If that is the case, I need to think is it possible to handle this, and old meters that are sending all data without ACK.

BR,
Mikko
BR,
Mikko

Hi Mikko,

The meter model and name is:

landis gyr e570

I also just figured something out, the messages that I just posted above have duplicate lines.
Edit: I've found that letting it run for some time makes the duplicate lines go away for some reason.

7E A8 87 CE FF 03 13 86
7E A8 87 CE FF 03 13 86

B7 0F 00 01 6E 7A 0C 07 E4 08 10 07 10 22 01 FF
B7 0F 00 01 6E 7A 0C 07 E4 08 10 07 10 22 01 FF

80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00
80 00 00 02 0E 01 0E 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 02 12 00

00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01
00 02 04 12 00 28 09 06 00 08 19 09 00 FF 0F 01 12 00 00 02 04 12 00 01

09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00
09 06 00 00 60 01 00 FF 0F 02 12 00 00 02 04 12 00 08 09 06 00 00 01 00

00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00
00 FF 0F 02 12 00 00 02 04 12 00 03 09 06 01 00 01 08 00 FF 0F 02 12 00

00 02 04 12 00 03 09 06 01 00 01 07 00 FF 61 71 7E
00 02 04 12 00 03 09 06 01 00 01 07 00 FF 61 71 7E

But again, the core of my

But again, the core of my question is: Do I have to manually translate this myself or is there a script which turns this message into OBIS code XML text?

It appears that the script just executes the exception for the translation part of the receive function and just prints out the raw data instead of translating it.

Hi,

You can use GXDLMSClient to get the data or GXDLMSTranslator to translate it to the XML. The problem here is that you don't have all the data. In your post, you have only the first frame. If you can receive all the frames, then there is no problem, but because you don't have all the data before you try to parse it, you get this error.

Error: Not enough data. 2 rows are missing. -->

BR,
Mikko

Ah ok I understand. The

Ah ok I understand. The Landis Gyr measurement device is not connected to any load and isn't counting any Wh, would this be the problem or is it commonly a hardware problem with the Raspberry/MbusSlave device?

Hi,

No, the problem is that all the data can't fit into one frame. You have received the first frame and I'm wondering why where are the other frames and why the meter is not sending them. The one reason might be that the meter is expecting the client sends acknowledge message before the meter sends the next frame.

BR,
Mikko

Hi,

Ok I will look into that. The energy company might give us some insight on the setup of the meter device.
Thank you for your help, when I've looked into it I will post here again :)

regards,
Hallmar

Hello Mikko,

I've looked at a configuration manual for Landis and Gyr devices and I'm wondering if you could give some insight into how it should be configured for push data. There is the trace settings shown in the picture below and also settings panel on HDLC and COSEM which says:

Select the required protocol for the planned activity in the "dlms Link
layer protocol" drop down list. Possible settings:
- HDLC, if the HDLC protocol must be used
- HDLC via IEC mode E (default), if the IEC protocol must be used
for opening the communication
- COSEM Wrapper, if the COSEM Wrapper over the TCP protocol
must be used (irrevelant if using MBus like we are doing)

Image

Hi,

If you can get a hex trace of how Map120 is handling this I can solve the correct settings from the byte stream. I need only trace from push messages.
BR,
Mikko

Hi,

Is the hex trace that I posted at the start of this thread not enough for this?

If not then I will ask our guy from the energy company to give us the hex trace from the meter :)

-Hallmar

Hi,

No, I need all the frames that the meter and client app sends to check all the frames.

BR,
Mikko

Hello Mikko,

I've had a meeting with the man from the energy company and he says that the hex trace that I've shown is the hex trace that the meter is sending via a push message over the M-Bus, there is nothing else to the message.

We tried reducing the objects that are sent via push messages and the same error came about 'not enough data'. The last step that we took was to remove the 'object list' message in the push message and at that point the script still just gave me the raw hex stream BUT right now I can read in the XML code what the Watt hours are and the 'error not enough data' is gone. In the example below that is
4D3B in hex which is 19771Wh in decimal, and that fits!

So right now I'm at a point where I just need to be able to translate that raw bytestream to XML and then I can write some python code to parse the XML code and get the Watthours value.

Hex stream(no object list):
7E A8
56 CE FF 03 13 F5 43 0F 00 03 AC 81 0C 07 E5 06 0A 04 0E 22 05 FF 80 00 00 02 0A 09 08 31 31 30 31 38 33 39 38 09 0C 07 E5 06 0A 04 0E 22 05 FF 80 00 00 14 00 00 00 00 00 00 4D 3B 06 00 00 01 78 12 00 E7 12 00 00 12 00 00 12 00 02 12 00 00 12 00 00 A2 C1 7E

XML(no object list):

<DataNotification>

<LongInvokeIdAndPriority Value="0003AC81" />

<DateTime Value="07E5060A040E2205FF800000" />
<NotificationBody>
<DataValue>
<Structure Qty="0A" >

<OctetString Value="3131303138333938" />

<OctetString Value="07E5060A040E2205FF800000" />
<Int64 Value="0000000000004D3B" /> //watt hours
<UInt32 Value="00000178" />
<UInt16 Value="00E7" /> //L1 voltage
<UInt16 Value="0000" />
<UInt16 Value="0000" />
<UInt16 Value="0002" /> //L1 current
<UInt16 Value="0000" />
<UInt16 Value="0000" />
</Structure>
</DataValue>
</NotificationBody>
</DataNotification>
7E

Hi,

There are several ways how meters are configured. Some meters are sending Push object list as a first parameter. This is nice because now the client knows what kind of data meter is sending. The bad side is that the amount of the sent bytes is bigger.

We have added getPushValues method to GXDLMSPushSetup object in Java and C#. It's released to Python and other programming languages at the following releases.

You can add object to the push object manually if meter is not sent the push object list as a first parameter. Somethins like this:

GXDLMSPushSetup push = new GXDLMSPushSetup();
push.PushObjectList.Add(new KeyValuePair<GXDLMSObject, GXDLMSCaptureObject>(new GXDLMSClock(), new GXDLMSCaptureObject(2, 0)));
foreach (KeyValuePair<GXDLMSObject, GXDLMSCaptureObject> it in push.GetPushValues(client, (List<object>)notify.Value))
{
int index = it.Value.AttributeIndex - 1;
Console.WriteLine(((IGXDLMSBase)it.Key).GetNames()[index] + ": " + it.Key.GetValues()[index]);
}

BR,
Mikko

Hello Mikko!

The push object list is no problem, I know in which order the object list is so all I need is to get the raw hex stream to be translated to XML just like how I do it manually with the Gurux GXDLMSDirector(translate message and take value parameter -> put value parameter and translate from PDU to XML) , I've tried using the translator.dataToXml() function but it doesn't translate it at all.

How would I go about this with the library? so for example taking this simple push message and translating it to XML?

7E A8
56 CE FF 03 13 F5 43 0F 00 03 AC 81 0C 07 E5 06 0A 04 0E 22 05 FF 80 00 00 02 0A 09 08 31 31 30 31 38 33 39 38 09 0C 07 E5 06 0A 04 0E 22 05 FF 80 00 00 14 00 00 00 00 00 00 4D 3B 06 00 00 01 78 12 00 E7 12 00 00 12 00 00 12 00 02 12 00 00 12 00 00 A2 C1 7E

Hi,

Meter says that there are more frames coming. It's interesting because it seems that all the data can fit into one frame. If this is a meter issue you and there is no more data coming, you can do it something like this:

GXByteBuffer data = new GXByteBuffer(ADD received data to here);
GXByteBuffer pdu = new GXByteBuffer();
GXDLMSTranslator t = new GXDLMSTranslator(TranslatorOutputType.SimpleXml);
t.FindNextFrame(data, pdu);
Console.WriteLine(t.PduToXml(pdu));

This is a simple example and you need to improve it for your use.

BR,
Mikko

Forum Rules