Repository navigation
extmod,rp2: Keep LWIP timer running if lwip poll_sockets() is called. - #18805
Conversation
|
@dpgeorge Unfortunately I still don't really know how we handle this for a hypothetical future "non-blocking getaddrinfo()" as previous discussed. Probably for this we would need to reference count pending DNS queries and use this as an extra condition for keeping the timer alive. EDIT: However there are other problems with a hypothetical async version, such as the current code assumes at most one outstanding DNS query. So can probably tackle if/when we write the async version. |
|
Code size report: |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #18805 +/- ##
=======================================
Coverage 98.42% 98.42%
=======================================
Files 174 174
Lines 22331 22331
=======================================
Hits 21980 21980
Misses 351 351 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Thanks for fixing this. The solution here looks good, although it would have been ideal if we could have somehow queried lwIP's active UDPs like we do with TCPs. But as you say, that won't work because of permanent UDP pbufs 😞 So I think the approach here is the next best thing. Regarding tests: I realised that multi tests can work just fine with only a single instance, and could write the following "multi" test: import socket, network
def instance0():
wlan = network.WLAN()
wlan.active(0)
try:
socket.getaddrinfo('micropython.org', 80)
except OSError as er:
print(repr(er))
wlan.active(1)
try:
socket.getaddrinfo('micropython.org', 80)
except OSError as er:
print(repr(er))
wlan.active(0)You can run that with |
|
Thanks for the thorough analysis and the cleaner fix, @projectgus. I agree #18805 is the right root-cause fix — keeping the timer alive so DNS timeouts actually work is much better than working around it at the On the fast-failure question — my use case is a 24/7 IoT device where WiFi drops and reconnects periodically. Application-level retry logic (priority queue with exponential backoff) handles reconnection. From that perspective, fast-failing DNS when WiFi is known-down is genuinely useful: waiting ~14s for lwIP's internal DNS timeout when there's no link wastes time the application could spend reconnecting WiFi. But I can see how on a device with a flaky WiFi signal, the link might recover mid-DNS-timeout, and lwIP's retries could succeed without the application needing to know. The idea of clearing DNS server settings on If you go that route, the combination of #18805 (timer stays alive → DNS timeouts work) plus DNS cleanup on disconnect (→ fast failure when interface is explicitly taken down) would cover both the indefinite-hang bug and the "don't waste 14s when WiFi is gone" scenario cleanly. Happy to close #18801 in favour of that approach. |
3d9ed78 to
4cff0a5
Compare
|
Thanks @dpgeorge, have added the test code you suggested to the PR and verified it fails on rp2 without this fix, and passes on both rp2 and esp32 ports otherwise. Thanks @paul-matthews for the feedback. Will have a look at adding a fix so DNS is cleared when Wi-Fi disconnects from the MicroPython side. It's hard to know what to do if Wi-Fi disconnects (perhaps temporarily) from the other side, @dpgeorge is keen to add a non-blocking getaddrinfo variant and that might be the best in the long term (i.e. if you detect Wi-Fi disconnected while waiting for a DNS response then your application can abort whatever it's doing and go back to wait for Wi-Fi to connect.) I'll look at putting up a PR to handle the case of failing immediately if the Python code has disconnected or deactivated the interface, though. |
|
|
||
|
|
||
| def instance0(): | ||
| wlan = network.WLAN() |
There was a problem hiding this comment.
I ran this test as per ./run-multitests.py -t a0 multi_wlan/*.py (against a RPI_PICO_W and standard unix micropython) and it didn't work as expected. Namely, it would pass without the fix from this PR.
That's because multi_wlan/01_ap_sta.py runs first and enables the AP interface without disabling it (because the test is skipped, but before it knows it should be skipped it's already started the AP). Since AP is active, that keeps the lwIP timer active.
Solution which worked for me was to add a line here (at the start of instance0) which disables the AP:
network.WLAN(network.WLAN.IF_AP).active(0)
(may want to guard that by if hasattr(network.WLAN, "IF_AP"): to be careful)
There was a problem hiding this comment.
Ah right, I had run multi_wlan/*.py but only with two instances. Fixed now.
I'd also been relying on the default argument of network.WLAN(), but this isn't documented to return any particular interface so I made it explicit.
There was a problem hiding this comment.
Thanks for updating. I tested it:
- without the fix in this PR, running all the
multi_wlantests over and over, the one added here fails consistently - with the fix, it passes consistently
4cff0a5 to
641aeb3
Compare
Fixes rp2 issue where socket.getaddrinfo() could block indefinitely if an interface goes down and still has a DNS server configured, as the LWIP timer stops running and can't time out the DNS query. Adds a regression test under multi_wlan that times out on rp2 without this fix. Fixes issue micropython#18797. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>
641aeb3 to
bfc69db
Compare
Summary
Fixes rp2 issue where
socket.getaddrinfo()could block indefinitely if an interface goes down and still has a DNS server configured, as the LWIP timer stops running (since #17272) and can't time out the DNS query.Fixes #18797.
This work was funded through GitHub Sponsors.
Testing
Was going to commit a regression test for this, but haven't found a good approach - the current
net_inettests don't know which interface is in use so they can't disable it, and the othermulti_wlantests don't assume internet connectivity. Suggestions welcome!Manual test code is simple:
Hangs indefinitely without this fix. Fails after the DNS timeout with this fix.
Trade-offs and Alternatives
A different fix was submitted by @paul-matthews as extmod/modlwip: Fix getaddrinfo blocking indefinitely without network. #18801 (thanks!) Compared to that PR, this PR should have smaller code size impact and may help LWIP's internal accounting if there's other places MicroPython blocks in a loop calling
poll_sockets()(unconfirmed).The other PR also immediately fails
getaddrinfo()if nonetifhas the link up. I'm not sure if this is desirable behaviour or not, as there are cases where the link can "flap" (i.e. poor Wi-Fi signal, other end of network cable power cycles, etc) and in those cases the internal LWIP DNS retries may actually succeed. I think another way to implement "fast failures" may be to have eitherwlan.disconnect()orwlan.active(False)clear the DNS server setting (and/or release DHCP). Curious what you think, @paul-matthews?I also tried changing the "stop timer" logic to keep the timer running if any UDP sockets were active (currently this check is for active TCP sockets only). This would cover DNS and other possible UDP activity. However, this approach isn't viable as LWIP DHCP currently keeps a UDP pbuf registered even when the interface is down, and the LWIP mDNS resolver permanently registers a single static UDP pbuf on init and never removes it.