extmod/modssl_mbedtls: Implement SSLSession support. by DvdGiessen · Pull Request #12780 · micropython/micropython

DvdGiessen · 2023-10-23T16:40:36Z

Summary

This implements support for the SSLSession class, introduced in CPython in 3.6 (see #2415). It allows saving session data from an active TLS client-side connection and then creating a new connection re-using this session data. Benefits include a faster handshake and reduced data usage for short connections.

Implementation details

This PR adds the SSLSession class, the session= parameter for the SSLContext.wrap_socket() method, and the session attribute for an SSLSocket object.

Additionally, I've added a non-standard part: The SSLSession.serialize() function that converts the session to a bytes object (also available via the buffer protocol, so perhaps exposing this function is redundant); so that it can be stored by the user, and a constructor for the SSLSession object that accepts a bytes-like object to reconstruct the session object (CPython doesn't allow direct construction). This allows storing the session somewhere and use it after a deep sleep or reboot.

The second commit adds server-side support for TLS tickets in the Unix port, so that we can meaningfully test the session resumption in tests. The third commit adds a test which tests session resumption using the SSLSession object, checking that the resumption worked by checking that a resuming consumes less data.

micropython/micropython-lib#829 is a companion MR that implements support in the ssl module wrapper. It is required for the tests to pass.

Usage example

A small example test, using a wrapper class around the TCP socket so we can count how many bytes of data we're sending/receiving:

from io import IOBase
import socket
import ssl
import time

import network
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect('my-ssid', 'my-password')
while wlan.ifconfig()[0] == '0.0.0.0':
    time.sleep(0.1)

class AccountingStream(IOBase):
    def __init__(self, stream):
        self.stream = stream
        self.bytes_read = 0
        self.bytes_written = 0
        for attr in dir(stream):
            if not hasattr(self, attr):
                value = getattr(stream, attr)
                if callable(value):
                    setattr(self, attr, value)
    def read(self, size=None):
        result = self.stream.read() if size is None else self.stream.read(size)
        self.bytes_read += len(result)
        return result
    def readinto(self, buf, nbytes=None):
        result = self.stream.readinto(buf) if nbytes is None else self.stream.readinto(buf, nbytes)
        self.bytes_read += result
        return result
    def write(self, buf):
        self.bytes_written += len(buf)
        return self.stream.write(buf)

def connect_and_count(host, port, session=None):
    addr = socket.getaddrinfo(host, port)[0][-1]
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.settimeout(10)
    sock.connect(addr)
    sock_accounting = AccountingStream(sock)
    sock_tls = ssl.wrap_socket(sock_accounting, cert_reqs=ssl.CERT_NONE, server_hostname=host, session=session)
    sock_tls.write('HEAD / HTTP/1.1\r\nHost: {}\r\n\r\n'.format(host).encode())
    print('Response:', sock_tls.readline())
    print('Bytes read and written:', sock_accounting.bytes_read, sock_accounting.bytes_written)
    session = sock_tls.session
    sock.close()
    return session

host, port = 'tls-v1-2.badssl.com', 1012
socket.getaddrinfo(host, port)

print('Clean start:')
t = time.ticks_ms()
session = connect_and_count(host, port, None)
print('Time (ms):', time.ticks_diff(time.ticks_ms(), t))

print('\nReusing SSLSession:')
t = time.ticks_ms()
session = connect_and_count(host, port, session)
print('Time (ms):', time.ticks_diff(time.ticks_ms(), t))

print('\nUsing serialized and parsed SSLSession:')
t = time.ticks_ms()
session = connect_and_count(host, port, ssl.SSLSession(bytes(session)))
print('Time (ms):', time.ticks_diff(time.ticks_ms(), t))

Clean start:
Response: b'HTTP/1.1 200 OK\r\n'
Bytes read and written: 4947 453
Time (ms): 1829

Reusing SSLSession:
Response: b'HTTP/1.1 200 OK\r\n'
Bytes read and written: 438 602
Time (ms): 919

Using serialized and parsed SSLSession:
Response: b'HTTP/1.1 200 OK\r\n'
Bytes read and written: 438 602
Time (ms): 926

Testing

I've deployed this in production and been running it for a number of years on a large number of devices.

github-actions · 2023-10-23T16:58:05Z

Code size report:

Reference:  unix/README: Update the supported targets list. [d901e98]
Comparison: extmod/modtls_mbedtls: Test SSLSession reuse. [merge of 1aeca3e]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64: +6280 +0.729% standard[incl +192(data)]
      stm32:    +0 +0.000% PYBV10
      esp32: +3340 +0.190% ESP32_GENERIC[incl +168(data)]
     mimxrt:    +0 +0.000% TEENSY40
        rp2: +1304 +0.141% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

codecov · 2023-10-23T17:13:56Z

Codecov Report

❌ Patch coverage is 90.90909% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.57%. Comparing base (1c3c201) to head (d221ba5).

Files with missing lines	Patch %	Lines
extmod/modtls_mbedtls.c	90.90%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #12780      +/-   ##
==========================================
- Coverage   98.58%   98.57%   -0.02%     
==========================================
  Files         182      182              
  Lines       23322    23375      +53     
  Branches        5        5              
==========================================
+ Hits        22993    23041      +48     
- Misses        328      333       +5     
  Partials        1        1

Flag	Coverage Δ
unix-coverage-32bit	`98.57% <90.90%> (-0.02%)`	⬇️
unix-coverage-64bit	`98.50% <90.90%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

projectgus · 2024-03-07T23:49:11Z

This is an automated heads-up that we've just merged a Pull Request
that removes the STATIC macro from MicroPython's C API.

See #13763

A search suggests this PR might apply the STATIC macro to some C code. If it
does, then next time you rebase the PR (or merge from master) then you should
please replace all the STATIC keywords with static.

Although this is an automated message, feel free to @-reply to me directly if
you have any questions about this.

DvdGiessen · 2024-03-19T15:16:07Z

Updated on latest master branch, added server-side support for TLS tickets to the Unix port, and added a test that checks (a) that SSLSession works and (b) that session resumption actually results in decreased data usage.

I've been using various versions of this patch for almost a year now to resume HTTPS connections without any trouble (though that might just be because I didn't try with many different configurations).

Marked as ready for review.

EDIT: And re-pushed because I forgot to add the documentation commit.

ccrighton · 2024-06-08T00:58:49Z

Hi Daniël,

Do you plan to update the asyncio implementation to use this functionality? If that was done it would be a minor effort to add SSL session support to many micropython web libraries.

Currently, we session set up times in the order of 5 seconds on the PICO W, the lack of SSL session reuse is a showstopper for web apps using asyncio's new SSL support.

Cheers,

Charlie

optimal-system · 2024-06-08T14:02:01Z

Hi Daniël,
Micro-controllers like the Rasperry Pico have a very interesting potential for IoT. To be controlled by a web interface they need a http library like Phew. To run the main program (to input data) and the web server (to output data) they need asyncio like the Phew implementation by ccrighton. To output data in a safe mode they also need httpS support. But when you put all that together it becomes very very sluggish ! (to toggle the built in LED it normally takes 25ms but with https and asyncio it takes more than 5000ms). All that to say there is very good reason to update the asyncio implementation...

DvdGiessen · 2024-08-04T14:08:48Z

I didn't look into how to use this with asyncio before. It appears that in CPython there is no interface to use SSLSessions with asyncio.

For the core support we'd need to for example add a session= parameter to asyncio.open_connection (here), similar to the server_hostname parameter. That would be a non-standard addition, so I'd prefer someone that's a bit more familiar with the asyncio ecosystem comments on whether this or some other solution would be preferable here.

For that reason I might prefer to split this into a separate follow-up PR, since it doesn't impact the changes proposed here and thus does not need to block considering / reviewing / merging this PR.

ccrighton · 2024-08-04T22:10:48Z

@DvdGiessen Hi Daniël,

I agree that the change is best handled as a separate PR as it requires changing the asyncio implementation to add session to the wrap_socket call.

Cheers,

Charlie

vshymanskyy · 2026-06-12T11:05:36Z

Wondering what's the plan on this one, it's an important addition

DvdGiessen · 2026-06-12T12:03:05Z

I'm still using it, which is why I occasionally update it to resolve merge conflicts, and the MR is ready for review.

Note #17074 was recently merged, which implements the PSK support. I haven't had time to look at it and don't know exactly what's possible with mbedTLS, but I think that supporting TLS 1.3 session resumption with a PSK would be nice to have, maybe even with the same interface so it works regardless of whether you're connecting to 1.2 or 1.3 peers.

dpgeorge · 2026-06-12T12:48:28Z

Thanks for keeping this PR active. It looks good and I've put it on the milestone for the next release.

dpgeorge

Thanks for the contribution here. I've now done an initial review and it's looking pretty good.

I made some comments inline. But i also had a bigger-picture thought about how the feature is implemented.

Considering that we don't need to be compatible with CPython (this is the tls module which is MicroPython-specific), it's not actually necessary to have the SSLSession object at all. It can just be a bytes object and always serialized. So, doing ssl_sock.session will return the serialized session as a bytes, and you pass that bytes back when you want to resume.

Benefits of that approach:

simpler, smaller API surface
less code, probably smaller firmware
there's no separate serialize/unserialize step
resource management is easier (wrt to calling mbedtls_ssl_session_free())

Drawbacks:

probably takes longer to serialize/unserialize every time you want to resume a session (although, if you anyway are serializing between connections there's no difference)
not possible in the future to add extra methods/attributes to the SSLSession (because it's just a bytes)

What do you think?

dpgeorge · 2026-06-12T12:56:17Z

+}
+
+static const mp_rom_map_elem_t ssl_session_locals_dict_table[] = {
+    { MP_ROM_QSTR(MP_QSTR_serialize), MP_ROM_PTR(&ssl_session_serialize_obj) },


Since it's possible to use bytes(session) to serialize it, I don't think this method is needed. It just means there are two ways to do the same thing, and the Zen of Python says there should ideally be only one obvious way.

In your use of this feature, do you find one of these ways more convenient than the other?

dpgeorge · 2026-06-12T13:00:29Z

+        }
+    } else if (dest[1] != MP_OBJ_NULL) {
+        // Store attribute.
+        if (attr == MP_QSTR_session) {


Is it necessary to be able to store to ssl_socket.session to set the session? Or is it enough to just pass the session through to wrap_socket()? Or are there situations where both of these ways of setting the session are needed?

Note that we do not need to match CPython here, because this is now the tls module. So we are free to design the API however we like. That said, if it's easy and efficient to match CPython, then that's definitely preferable.

dpgeorge · 2026-06-12T13:12:35Z

+    mp_buffer_info_t bufinfo;
+    mp_get_buffer_raise(args[0], &bufinfo, MP_BUFFER_READ);
+
+    mp_obj_ssl_session_t *self = m_new_obj(mp_obj_ssl_session_t);


Does this type need a finalizer to call mbedtls_ssl_session_free()? It looks like mbedtls does some internal allocations using mbedtls_calloc() when populating a session. And they will need to be freed manually.

dpgeorge · 2026-06-12T13:13:14Z

    }

+    if (ssl_session != mp_const_none) {
+        mp_obj_ssl_session_t *session = MP_OBJ_TO_PTR(ssl_session);


This probably needs a type check to make sure the session really is a SSLSession.

dpgeorge · 2026-06-12T13:14:20Z

+        // Load attribute.
+        if (attr == MP_QSTR_session) {
+            mp_obj_ssl_session_t *o = m_new_obj(mp_obj_ssl_session_t);
+            o->base.type = &ssl_session_type;


Can use the helper mp_obj_malloc(mp_obj_ssl_session_t, &ssl_session_type) here.

dpgeorge · 2026-06-12T13:17:10Z

+    os.stat(keyfile)
+except OSError:
+    print("SKIP")
+    raise SystemExit


This check can go. All the multi-net tests assume that the cert files are available.

dpgeorge · 2026-06-19T03:20:54Z

@Gadgetoid I think this feature might be of interest to you.

dpgeorge · 2026-08-13T01:00:18Z

@DvdGiessen any comments on the above review? This is a good feature to have, just need to consider the overall API approach.

DvdGiessen · 2026-08-13T08:03:44Z

Sorry, this slipped of my radar. Thanks for for the feedback! I agree it would indeed make the code simpler, I'll see about making the changes when I find some time.

Signed-off-by: Daniël van de Giessen <[email protected]>

DvdGiessen force-pushed the mbedtls_sslsession branch from 57c5d78 to 43824ae Compare October 23, 2023 16:47

DvdGiessen marked this pull request as draft October 23, 2023 16:55

dpgeorge added the extmod Relates to extmod/ directory in source label Nov 3, 2023

DvdGiessen force-pushed the mbedtls_sslsession branch from 43824ae to 53bb552 Compare February 20, 2024 17:35

Gadgetoid mentioned this pull request Feb 29, 2024

global: Remove the STATIC macro. #13763

Merged

DvdGiessen force-pushed the mbedtls_sslsession branch from 53bb552 to e529117 Compare March 4, 2024 17:07

DvdGiessen force-pushed the mbedtls_sslsession branch 2 times, most recently from f014564 to 6c50ae1 Compare March 19, 2024 14:38

DvdGiessen mentioned this pull request Mar 19, 2024

ssl: Implement SSLSession support. micropython/micropython-lib#829

Open

DvdGiessen force-pushed the mbedtls_sslsession branch from 6c50ae1 to 9a48098 Compare March 19, 2024 14:58

DvdGiessen marked this pull request as ready for review March 19, 2024 15:11

DvdGiessen force-pushed the mbedtls_sslsession branch 3 times, most recently from caeb380 to feae3a7 Compare March 20, 2024 12:57

DvdGiessen force-pushed the mbedtls_sslsession branch from feae3a7 to a7c1dc6 Compare May 27, 2024 12:36

DvdGiessen force-pushed the mbedtls_sslsession branch from a7c1dc6 to 06c5929 Compare August 31, 2024 23:08

DvdGiessen force-pushed the mbedtls_sslsession branch from 06c5929 to 30ce6ac Compare October 24, 2024 20:08

DvdGiessen force-pushed the mbedtls_sslsession branch from 30ce6ac to 95e5585 Compare November 5, 2024 23:36

DvdGiessen force-pushed the mbedtls_sslsession branch from 95e5585 to 4f91e1c Compare December 6, 2024 10:25

DvdGiessen force-pushed the mbedtls_sslsession branch from 4f91e1c to 2e1b344 Compare January 14, 2025 21:29

DvdGiessen force-pushed the mbedtls_sslsession branch from 2e1b344 to 16223e9 Compare April 16, 2025 12:02

DvdGiessen force-pushed the mbedtls_sslsession branch from 16223e9 to 4f8529d Compare May 14, 2025 14:34

DvdGiessen force-pushed the mbedtls_sslsession branch from 4f8529d to 3eca373 Compare May 20, 2025 14:51

DvdGiessen force-pushed the mbedtls_sslsession branch from 3eca373 to 70df1af Compare August 4, 2025 14:32

DvdGiessen force-pushed the mbedtls_sslsession branch from 70df1af to 88c4c9c Compare December 10, 2025 10:46

DvdGiessen force-pushed the mbedtls_sslsession branch from 88c4c9c to d43cbc0 Compare March 25, 2026 17:12

DvdGiessen force-pushed the mbedtls_sslsession branch from d43cbc0 to 1aeca3e Compare June 12, 2026 08:48

dpgeorge added this to the release-1.29.0 milestone Jun 12, 2026

dpgeorge reviewed Jun 12, 2026

View reviewed changes

dpgeorge modified the milestones: release-1.29.0, release-1.30 Aug 13, 2026

DvdGiessen added 5 commits August 20, 2026 14:04

extmod/modtls_mbedtls: Implement SSLSession support.

1bbe6d5

Signed-off-by: Daniël van de Giessen <[email protected]>

lib/micropython-lib: Update to include SSLSession support.

240cbc8

Signed-off-by: Daniël van de Giessen <[email protected]>

extmod/modtls_mbedtls: Documentation of SSLSession support.

5386281

Signed-off-by: Daniël van de Giessen <[email protected]>

extmod/modtls_mbedtls: Add server-side support for TLS tickets.

20e3cd0

Signed-off-by: Daniël van de Giessen <[email protected]>

extmod/modtls_mbedtls: Test SSLSession reuse.

d221ba5

Signed-off-by: Daniël van de Giessen <[email protected]>

DvdGiessen force-pushed the mbedtls_sslsession branch from 1aeca3e to d221ba5 Compare August 20, 2026 12:05

Conversation

DvdGiessen commented Oct 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Implementation details

Usage example

Testing

Uh oh!

github-actions Bot commented Oct 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Oct 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

projectgus commented Mar 7, 2024

Uh oh!

DvdGiessen commented Mar 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ccrighton commented Jun 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

optimal-system commented Jun 8, 2024

Uh oh!

DvdGiessen commented Aug 4, 2024

Uh oh!

ccrighton commented Aug 4, 2024

Uh oh!

vshymanskyy commented Jun 12, 2026

Uh oh!

DvdGiessen commented Jun 12, 2026

Uh oh!

dpgeorge commented Jun 12, 2026

Uh oh!

dpgeorge left a comment

Choose a reason for hiding this comment

Uh oh!

dpgeorge Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

dpgeorge Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

dpgeorge Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

dpgeorge Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

dpgeorge Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

dpgeorge Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

dpgeorge commented Jun 19, 2026

Uh oh!

dpgeorge commented Aug 13, 2026

Uh oh!

DvdGiessen commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

DvdGiessen commented Oct 23, 2023 •

edited

Loading

github-actions Bot commented Oct 23, 2023 •

edited

Loading

codecov Bot commented Oct 23, 2023 •

edited

Loading

DvdGiessen commented Mar 19, 2024 •

edited

Loading

ccrighton commented Jun 8, 2024 •

edited

Loading