cooking errors

[jdavid]

There're a number of frames that cannot be parsed in the Pi@Finse. For example:

In [13]: frame = base64.b64decode(b'PD0+hgMjNzc0QzY3MDU3QzEwNTQxMyNub2RlX2lkIzQzI01BQzo0MTA1RDRDNSNBQ0M6Nzg7LTE5Mzs5OTIjQkFUOjEwMCM=')
    ...: print(frame)
    ...: parse_frame(frame)
    ...: 
b'<=>\x86\x03#774C67057C105413#node_id#43#MAC:4105D4C5#ACC:78;-193;992#BAT:100#'
Warning: text frames not supported (134)

We should resolve all these so the queue is empty.

For instance, in the example above, it's a text frame. Support for parsing text frames was removed a long time ago. Should we support text frames?

[jdavid]

There're 1114 frames with errors.

• Change the code so the consumer can define the action to take in a failure: nack, nack without requeue, or nothing.

• Discard text frames, send nack without requeue. These should become dead letters.

• Handle the remaining failures.

[jdavid]

Right now there are 188 frames in the wsn_raw_cook.

One received the 2019/10/01 looks as it's somehow corrupt:

{"received": 1569889239, "id": "rx", "reserved": "//4=", "source_addr": "ABOiAEFrTtc=", "options": "wQ==", "data": "AACsAAAArQAAAK4AAACvAO1P/////f19hk5GIRhCGMwhhgQwYMGDAgYEBAQMBAIGAIEBjGNmGMYYYYYIGDDAwA=="}

To resolve this one we will need to reach for the SD card. This is the same but cooked:

{'received': 1569889239, 'reserved': b'\xff\xfe', 'source_addr': 5526146538753751, 'data': b'\x00\x00\xac\x00\x00\x00\xad\x00\x00\x00\xae\x00\x00\x00\xaf\x00\xedO\xff\xff\xff\xfd\xfd}\x86NF!\x18B\x18\xcc!\x86\x040`\xc1\x83\x02\x06\x04\x04\x04\x0c\x04\x02\x06\x00\x81\x01\x8ccf\x18\xc6\x18a\x86\x08\x180\xc0\xc0', 'options': b'\xc1', 'id': 'rx'}

We had as well 2 frames in the wsn_data_django queue. These had a time value too large to fit into the database, and produced a 500 error. We have:

• Improved the error message logged in the Pi
• Added a check in the server to discard frames where the time field is too large