Long Distance 2

This weekend I participated in the Real World CTF with WreckTheLine. Unlike many other CTFs, these challenges were all based on real applications and systems. It’s interesting being able to use (and gain) domain knowledge, and while contrived challenges are fun, exploiting a system that exists in the real world – on the real internet – is another level of engagement.

Challenge

This misc challenge consisted of a 290 MB WAV file named 486_375MHz-1MSps-1MHz.wav, an 8 MB flash_dump, and a mandate to recover the information contained in the transmission:

Of late, whispers doth persist behind mine back. Yesterday, under the studio tower, a peculiar contraption was found by me. I am most intrigued to discover the content of their discourse.

RF

The WAV file metadata showed 51 seconds, 2 channels, 24 bit resolution, and 1 MHz sample rate. Audibly, it was a mostly silent with several bursts of data, sounding like a pop followed by buzzing.

Audacity was in the mood for 3 hours of plugin updates, so I used Sonic Visualiser, which showed the bursts of data in the waveform. Waveform showing two channels of silence followed by a short loud burst and a longer textured section, repeated twice Zooming in, things became more interesting - FM? Waveform zoomed in to show the individual squiggles getting closer together and farther apart over time in both channels We can use the spectrogram tool to see the distribution of energy (color) across frequencies (Y axis) over time (X axis), with an interesting result… Waveform for context, and spectrogram showing that the high energy pulse is zigzagging up and down in frequency during the data transmission And further into the data section, discontinuities in the frequency sweeps - Probably the symbols that we needed recover data from. Another part of the data transmission, where the spectrogram zigzag repeatedly jumps to a different part of the zig or zag

A teammate, qwertboy, had linked a writeup of last year’s Long Distance challenge by the organizers team, where they had a similar file containing a RF capture, and found the flag by decoding the LoRaWAN traffic.

What I was seeing matched signals they found. Both had similar patterns in the waveform and demodulated FM, and after setting up GNU Radio, in the constellation display.

Flash Image

My first port of call with an unknown firmware image is usually binwalk. In this case the signatures and strings it found led me to believe that this was an image for an ESP32, used mesh networking, and (incorrectly) that it might be for an IoT lightbulb or lighting controller.

$ binwalk flash_dump

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
88595         0x15A13         Neighborly text, "neighbors a simple (0 id) broadcast"
109607        0x1AC27         HTML document header
109988        0x1ADA4         HTML document footer
113954        0x1BD22         Neighborly text, "Neighbor Info module config: Ambient Lighting"
114583        0x1BF97         Neighborly text, "Neighbor Info module config: Ambient Lighting"
118036        0x1CD14         Neighborly text, "NeighborsKET from Node 0x%x to Node 0x%x (last sent by 0x%x)"

[...]
155668        0x26014         Unix path: /home/runner/.platformio/packages/framework-arduinoespressif32/libraries/SPI/src/SPI.cpp
173532        0x2A5DC         AES S-Box
[...]

Extracting the identified files with binwalk -e found assorted HTML and CSS, along with a JSON blob from 0x3A7000:

{
  "name": "Meshtastic",
  "short_name": "Meshtastic",
  "start_url": ".",
  "description": "Meshtastic web app",
  "icons": [
    {
      "src": "/icon.svg",
      "sizes": "any",
      "type": "image/svg+xml"
    }
  ],
  "theme_color": "#67ea94",
  "background_color": "#67ea94",
  "display": "standalone"
}

So I knew early that we’re working with Meshtastic, but didn’t yet understand LoRa vs LoRaWAN. Reading through the site found radio settings to help with decoding, the Meshtastic packet header format to help with parsing, and an overview of the encryption. I later found a diagram of a LoRa Meshtastic packet which was useful, but note the backwards flags field.

LoRa

I spent quite a while trying to get rpp0/gr-lora set up (it officially requires the long-dead python2) and working, and a while more seeing if tapparelj/gr-lora_sdr or its fork martynvdijke/gr-lora_sdr (as used by the AUR package) were any more cooperative. In the end, I replaced the python2-foo dependencies with python3-foo, and it didn’t complain. Still, the correct configuration for the decoder eluded me.

Saturday morning started with another teammate, tcode2k16, finding a potential signal decoding result before I’d finished breakfast. We individually chased our tails for a while on this, and it turned out to have two issues - The decoded data was incorrect, and we were confusing LoRa (physical layer) with LoRaWAN (a communication protocol built on LoRa) and trying to parse the payload as the latter.

We Need to Go Deeper

Once tcode was making progress on decoding and parsing, I went back to the firmware image. The source for this build, less any CTF modifications, is at meshtastic/firmware@v2.2.14.57542ce, and tcode found a default AES key there, but we should check for a custom one in flash.

To start, I dropped the image into a hex editor, which wasn’t particularly useful until I looked into how ESP-IDF partitioning works. The default partition table isn’t at the beginning of flash, but at 0x8000.

Rather than spend time parsing the partition table correctly, I guessed the layout based on the hex dump:

00008000  aa 50 01 02 00 90 00 00  00 50 00 00 6e 76 73 00  |.P.......P..nvs.|
00008010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00008020  aa 50 01 00 00 e0 00 00  00 20 00 00 6f 74 61 64  |.P....... ..otad|
00008030  61 74 61 00 00 00 00 00  00 00 00 00 00 00 00 00  |ata.............|
00008040  aa 50 00 10 00 00 01 00  00 00 25 00 61 70 70 00  |.P........%.app.|
00008050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00008060  aa 50 00 11 00 00 26 00  00 00 0a 00 66 6c 61 73  |.P....&.....flas|
00008070  68 41 70 70 00 00 00 00  00 00 00 00 00 00 00 00  |hApp............|
00008080  aa 50 01 82 00 00 30 00  00 00 10 00 73 70 69 66  |.P....0.....spif|
00008090  66 73 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |fs..............|

Much later I realized the partition table, human-readable with extra info, is also available right in the firmware repository as partition-table.csv. Here’s that version, to save at least one of us some squinting at hex:

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x009000, 0x005000,
otadata,  data, ota,     0x00e000, 0x002000,
app,      app,  ota_0,   0x010000, 0x250000,
flashApp, app,  ota_1,   0x260000, 0x0A0000,
spiffs,   data, spiffs,  0x300000, 0x100000,

“NVS” is used by the ESP-IDF Non-Volatile Storage Library, which seemed like a good place to store settings. Carve out 0x5000 bytes starting at 0x9000 in the image with dd if=flash_dump bs=$((0x1000)) skip=9 count=5 of=nvs.bin, and parse it with nvs_tool.py from the ESP-IDF repository: python nvs_tool.py -i nvs.bin. Unfortunately none of the data it held looked like a 128-bit or 256-bit AES key, or for that matter, any of the other configuration parameters I was expecting.

Where else would the firmware store its configuration? Looking back at the partition table, “nvs” was a bust, “otadata” seems update-related, “app” is probably the running code, “flashApp” may be for in-progress updates, or “spiffs”. The “SPI Flash FileSystem” sounded promising, so I extricated it with dd if=flash_dump bs=$((0x100000)) skip=3 count=1 of=spiffs.bin, but igrr/mkspiffs didn’t like it. file wasn’t familiar. hexdump showed this at the beginning of the file:

00000000  03 00 00 00 f0 0f ff f7  6c 69 74 74 6c 65 66 73  |........littlefs|

Ah. littlefs on a partition named spiffs. Probably some backwards-compatibility thing…

Regardless, the littlefs readme had a lot of useful tools. tniessen/littlefs-disk-img-viewer is a webapp with a live demo, which after some block size guessing (4096) was able to read the filesystem. littlefs viewer showing spiffs.bin open, that 121/256 blocks are used, and two folders inside named “prefs” and “static” static/ held the Meshtastic webui resources, and prefs/ had channels.proto, config.proto, and db.proto, which I downloaded.

Protobuf

Protobuf (protocol buffers) is a relatively common data interchange system, which Meshtastic uses for both preferences and over-the-air communication. At first I tried to use protoc with the Meshtastic protobuf definitions to produce Python code to read the files, and then protobufpal, but they weren’t parsing the prefs files, so in the interest of time I ended up using mildsunrise/protobuf-inspector.

$ protobuf_inspector <channels.proto
root:
    1 <chunk> = message:
        2 <chunk> = message:
            2 <chunk> = bytes (1)
                0000   01                                                                       .
        3 <varint> = 1
    1 <chunk> = message:
        1 <varint> = 1
        2 <chunk> = message:
            2 <chunk> = bytes (32)
                0000   CE F8 DB 8E 8E 60 17 FD 6D CC A2 1D B8 A1 47 6D 45 14 80 AC D7 F4 F9 F7  .....`..m.....GmE.......
                0018   69 A7 63 F5 28 C0 11 F7                                                  i.c.(...
            3 <chunk> = "Buddies"
            4 <32bit> = 0x00000001 / 1 / 1.40130e-45
        3 <varint> = 2
    1 <chunk> = message(1 <varint> = 2, 2 <chunk> = empty chunk)
    1 <chunk> = message(1 <varint> = 3, 2 <chunk> = empty chunk)
    1 <chunk> = message(1 <varint> = 4, 2 <chunk> = empty chunk)
    1 <chunk> = message(1 <varint> = 5, 2 <chunk> = empty chunk)
    1 <chunk> = message(1 <varint> = 6, 2 <chunk> = empty chunk)
    1 <chunk> = message(1 <varint> = 7, 2 <chunk> = empty chunk)
    2 <varint> = 22

The leading number associates a value with its attribute in the protobuf definition. The root seems to be an array of Channel objects, so within each of those we should be able to look for the attribute numbers in channel.proto. Channel->2 is a ChannelSettings settings, and ChannelSettings->2 is bytes psk. We’ve found our channel key!

Since then I’ve realized that prefs/channels.proto isn’t a Channel from channel.proto, it’s a ChannelFile from deviceonly.proto, so here’s the interesting parts in a more readable format:

{
  "channels": [
    {
      "index": 0,
      "settings": {...},
      "role": "PRIMARY"
    },
    {
      "index": 1,
      "settings": {
        "channel_num": 0,
        "psk": [
            206,248,219,142,142,96,23,253,109,204,162,29,184,161,71,109,
            69,20,128,172,215,244,249,247,105,167,99,245,40,192,17,247
        ],
        "name": "Buddies",
        "id": 1,
        "uplink_enabled": false,
        "downlink_enabled": false
      },
      "role": "SECONDARY"
    },
    [followed by 6 empty role: DISABLED channels]
  ],
  "version": 22
}

And the LoRa part of prefs/config.proto:

{
  "lora": {
    "ignore_incoming": [],
    "use_preset": true,
    "modem_preset": "LONG_FAST",
    "bandwidth": 0,
    "spread_factor": 0,
    "coding_rate": 0,
    "frequency_offset": 0,
    "region": "CN",
    "hop_limit": 3,
    "tx_enabled": true,
    "tx_power": 19,
    "channel_num": 66,
    "override_duty_cycle": false,
    "sx126x_rx_boosted_gain": true,
    "override_frequency": 0
  }
}

We no longer have to guess our LoRa settings, we can just copy the LONG_FAST parameters from the radio settings page!

Back to Decoding

With known radio settings we could revisit the GNU Radio flow graph with gr-lora. Here’s what I ended up with: download grc GNU Radio flow graph. Both channels from WAV file source are converted from floats to complex numbers, sent through a bypassed Throttle block, then to a disabled QT GUI Sink block and a LoRa Receiver block

And its output: full log

Bits (nominal) per symbol: 	5.5
Bins per symbol: 	2048
Samples per symbol: 	8192
Decimation: 		4
 2d 31 e0 bc 4b 6c fa c4 c0 6d fa 66 26 d2 02 0b 08 a7 92 b3 78 fb 63 77 d7 e0 54 d7 4f 67 1e c0 2d f1 8c 7d 04 66 c9 31 bb 22 40 0f c9 ec 25 c8 71 33 (Klmf&xcwTOg-}f1"@%q3)
[...]
 22 31 70 ff ff ff ff c4 c0 6d fa 5f 8a 54 22 02 04 a7 92 53 1f 89 a3 6f ea 30 18 c9 ce b7 e7 1f a3 cd 72 71 ed a7 3a (m_T"So0rq:)
 09 11 40/usr/include/c++/13.2.1/bits/stl_vector.h:1125: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = unsigned char; _Alloc = std::allocator<unsigned char>; reference = unsigned char&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.

The crash may have been because of my python2/python3 change. I hoped it wasn’t preventing decoding of the message with the flag, and it turned out fine.

This still looked a bit nonsensical consider the Meshtastic header format. When aligning the first part of each message, I would have expected to see repeated source and destination IDs, channel hash bytes, and padding, and flag bytes with the high nibble empty.

DEST_NODEID SRC_NODEID_ PKT_UNIQ_ID FL CH PAD__ DATA...
2d 31 e0 bc 4b 6c fa c4 c0 6d fa 66 26 d2 02 0b 08 a7 92 b3 78 fb ...
1b 31 e0 c4 c0 6d fa bc 4b 6c fa 29 5d 91 38 03 08 a7 92 12 19 7e ...
49 31 20 c4 c0 6d fa bc 4b 6c fa a6 41 be 1e 0b 08 a7 92 b9 4d 99 ...
1b 31 e0 bc 4b 6c fa c4 c0 6d fa 0f 21 39 44 03 08 a7 92 b8 56 c7 ...
5f 30 00 ff ff ff ff c4 c0 6d fa a3 f3 0f 12 03 08 a7 92 aa c1 8d ...
5f 30 00 ff ff ff ff c4 c0 6d fa a3 f3 0f 12 02 08 a7 92 aa c1 8d ...
...

There are some columns that do have the properties I would expect, though… Let’s shift the column headers over a byte at a time and see if they line up:

?? ?? ?? DEST_NODEID SRC_NODEID_ PKT_UNIQ_ID FL CH PAD__ DATA...
2d 31 e0 bc 4b 6c fa c4 c0 6d fa 66 26 d2 02 0b 08 a7 92 b3 78 fb ...
1b 31 e0 c4 c0 6d fa bc 4b 6c fa 29 5d 91 38 03 08 a7 92 12 19 7e ...
49 31 20 c4 c0 6d fa bc 4b 6c fa a6 41 be 1e 0b 08 a7 92 b9 4d 99 ...
1b 31 e0 bc 4b 6c fa c4 c0 6d fa 0f 21 39 44 03 08 a7 92 b8 56 c7 ...
5f 30 00 ff ff ff ff c4 c0 6d fa a3 f3 0f 12 03 08 a7 92 aa c1 8d ...
5f 30 00 ff ff ff ff c4 c0 6d fa a3 f3 0f 12 02 08 a7 92 aa c1 8d ...
...

That looks better, the fields all had the properties I expected. The leading 3 bytes ended up being the LoRa header, as seen in the diagram - Payload length, coding rate, CRC flag, header CRC, padding.

On to Parsing

Disassembling the headers in python was relatively straightforward, but decrypting the Meshtastic payload was aggravating. The encrypt and decrypt functions are the same for AES-CTR, and for us are in ESP32CryptoEngine.cpp. I should have looked at the relevant bits for other platforms, since they must all be compatible and others are more straightforward, but I didn’t. The problem boiled down to me chasing endianness issues before double checking that I was putting the parts of the nonce together in the wrong order or trying to decrypt other messages.

While I worked on building a custom Counter for PyCryptodome, tcode used my parsing and the python-mbedtls¹ library to successfully decrypt it. A few minutes later I got mine working, using the PyCryptodome, the default Counter, and the correct nonce ordering and message.

#!/usr/bin/env python3

from Crypto.Cipher import AES
from Crypto.Util import Counter


pkt = bytes.fromhex(
    "683060ffffffffc4c06dfa8899c2020304a7925001502e35d73ab793c3e9ad6acc9da6a4955bc56b8054e9989d76f5355cf2868b90bffae321d51077be4e1774fc074f63a4c0af6ba38f33a829b0784bdadbcc738b16e930e741c48f4d6c0cab012e5605122dce40eaeb7b7626"
)
psk = bytes.fromhex(
    "CEF8DB8E8E6017FD6DCCA21DB8A1476D451480ACD7F4F9F769A763F528C011F7"
)

lora_header = pkt[0:3]
has_crc = (lora_header[1] >> 4 & 1) > 0
lora_payload = pkt[3:]
lora_payload_crc = lora_payload[-2:] if has_crc else None
if has_crc:
    lora_payload = lora_payload[:-2]
lora_payload_len = lora_header[0]
if len(lora_payload) != lora_payload_len:
    print(
        "Warning: Incorrect payload length. expected {}, got {}".format(
            lora_payload_len, len(lora_payload)
        )
    )
mt_header = lora_payload[:16]
mt_dest = mt_header[0:4]
mt_src = mt_header[4:8]
mt_packet_id = mt_header[8:12]
mt_flags = mt_header[12]
mt_flags_hop_limit = mt_flags & 0b111
mt_flags_want_ack = (mt_flags >> 3 & 1) > 0
mt_channel_hash = mt_header[13]
mt_header_padding = mt_header[14:16]
mt_payload = lora_payload[16:]

nonce = mt_packet_id + b"\0" * 4 + mt_src
cipher = AES.new(psk, AES.MODE_CTR, initial_value=0, nonce=nonce)
mt_payload_decrypted = cipher.decrypt(mt_payload)

open("decrypt.out", "wb").write(mt_payload_decrypted)
print(mt_payload_decrypted)

Victory!

b'\x08\x01\x12Talright alright. the key is rwctf{No_h0p_th1s_tim3_c831bcad725935ba25c0a3708e49c0c8}'

Fully parsed:

{
  "portnum": "TEXT_MESSAGE_APP",
  "payload": "alright alright. the key is rwctf{No_h0p_th1s_tim3_c831bcad725935ba25c0a3708e49c0c8}"
}

Conclusion

Thanks to tcode for working with me on this challenge. At one point during decoding they said “I was doing something silly”. In my experience, that’s all of programming, CTFing, and many other technical endeavors. I hope this post was worth the read despite its length, but I think it’s important to not gloss over the challenges and mistakes instead of reinforcing anyone’s imposter syndrome.

WreckTheLine finished in 7th place of 2291 teams. This challenge was worth 320 points out of our 1662 (19%), and was solved by a total of 9 teams.

I would recommend against using Synss/python-mbedtls for real cryptography. It has basically no documentation I can find besides examples, and what is there (docstrings) is scary. For example, AES.new(key, mode, iv, ad) (note that iv=nonce+counter, effectively):

iv: The initialization vector (IV). The IV is required for every mode but ECB and CTR where it is ignored. If not set, the IV is initialized to all 0, which should not be used for encryption.

AES-CTR must never use the same (key, nonce, counter) twice or its security properties are broken. Luckily failing to specify an iv parameter does produce an error. Additionally, the ad (Associated Data / Additional Authenticated Data, I assume) parameter is not even in the docstring. ↩︎