Repository navigation
extmod/network_wiznet5k: Switch to upstream ioLibrary_Driver. - #19713
wiznet-grace wants to merge 3 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #19713 +/- ##
==========================================
+ Coverage 98.55% 98.59% +0.03%
==========================================
Files 182 182
Lines 23346 23346
Branches 5 5
==========================================
+ Hits 23009 23017 +8
+ Misses 336 328 -8
Partials 1 1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Code size report: |
There was a problem hiding this comment.
Thanks @wiznet-grace for splitting the big PR up and submitting this.
Generally this looks really good, we would like to use the latest upstream Wiznet library.
I have not done any testing yet, but I have some comments below about removing some of the patches we previously had in our MicroPython fork. Most of these were solving real problems that users experienced with MicroPython+Wiznet, so we don't want to have any regressions if we switch to the upstream.
| #endif | ||
| } | ||
| int8_t wizchip_socket(uint8_t sn, uint8_t protocol, uint16_t port, uint8_t flag) { | ||
| return socket(sn, protocol, port, flag); |
There was a problem hiding this comment.
One of the problems we ran into in the past is having ioLibrary_Driver provide these libc names (socket, close, listen, recv). This is a problem if we also link lwIP TCP/IP stack into the binary, for example (because the symbol name clashes).
Wrapping them like this in this file doesn't solve the problem, because the symbol socket is still defined by the ioLibrary_Driver module and it'll be part of the final linker input.
The previous approach we used for MicroPython was this one:
PR commit
Would you consider applying this change? Otherwise, you could add a simpler optional macro in ioLibrary_Driver that changes the names socket -> wizchip_socket on the library side if set. But the other name has to not be compiled into the binary, otherwise it doesn't solve the problem.
There was a problem hiding this comment.
EDIT: Fixed commit link
There was a problem hiding this comment.
Right, the wrappers didn't help. Fixed in the library instead: Wiznet/ioLibrary_Driver#187 (merged) adds an opt-in WIZCRight, the wrappers didn't help.
Fixed in the library instead: Wiznet/ioLibrary_Driver#187 (merged) adds an opt-in WIZCHIP_PREFIXED_EXPORTS that renames those functions to wizchip_* in the library itself. The submodule now includes it, the build sets it to 1 again, and the wrappers are gone. nm shows no unprefixed names left on any of the affected boards.
HIP_PREFIXED_EXPORTS that renames those functions to wizchip_* in the library itself. The submodule now includes it, the build sets it to 1 again, and the wrappers are gone. nm shows no unprefixed names left on any of the affected boards.
| CFLAGS_THIRDPARTY += -DWIZCHIP_PREFIXED_EXPORTS=1 | ||
| ifeq ($(MICROPY_PY_LWIP),1) | ||
| # When using MACRAW mode (with lwIP), maximum buffer space must be used for the raw socket | ||
| CFLAGS_THIRDPARTY += -DWIZCHIP_USE_MAX_BUFFER |
There was a problem hiding this comment.
This change was added to set the buffer sizes to 16KB when using lwIP, see this PR commit.
Looking at the upstream library I couldn't see any change to the macraw buffer size for W5200 (didn't check the others). Is this problem solved in some other way?
There was a problem hiding this comment.
EDIT: fixed commit link
There was a problem hiding this comment.
The driver handles this now. Before opening the MACRAW socket, wiznet5k_init() gives socket 0 all the buffer memory the other sockets would have had, so no library patch is needed:
// Configure buffers for fast MACRAW
#if _WIZCHIP_ < W5200
uint8_t sn_size[8] = {8, 0, 0, 0, 8, 0, 0, 0};
#elif (_WIZCHIP_ == W5200 || _WIZCHIP_ == W5300 || _WIZCHIP_ == W5500 || _WIZCHIP_ == W6100)
uint8_t sn_size[16] = {16, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0};
#elif (_WIZCHIP_ == W6300)
uint8_t sn_size[16] = {32, 0, 0, 0, 0, 0, 0, 0, 32, 0, 0, 0, 0, 0, 0, 0};
#endif
ctlwizchip(CW_INIT_WIZCHIP, sn_size);
The first half of the array is the TX size per socket and the second half the RX size, so socket 0 gets 16 KB each way on the W5500 and the other sockets get none.
| mp_hal_pin_high(wiznet5k_obj.cs); | ||
| } | ||
|
|
||
| void mpy_wiznet_yield(void) { |
There was a problem hiding this comment.
This yield function was added in the MicroPython fork to improve MicroPython performance when waiting on Wiznet chips in socket mode. PR commit here.
There doesn't seem to be any equivalent feature in the upstream library. Would you consider adding it? It doesn't change anything if the WIZNET_YIELD macro is not set.
There was a problem hiding this comment.
EDIT: Fixed commit link
There was a problem hiding this comment.
This is covered by the second commit: the driver opens sockets in non-blocking mode and waits in its own loop with mp_event_handle_nowait(), so Ctrl-C and callbacks still run (tested on W5100S/W5500). The one exception is the SENDOK wait after a UDP sendto() inside the library, which is bounded by the chip timeout (~1.8 s on W5500).
Point the submodule at WIZnet's ioLibrary_Driver instead of a fork that carries MicroPython-specific patches and has drifted from upstream. The driver is rewritten in the same commit, as the old one relies on the fork's WIZCHIP_EXPORT() macro. WIZCHIP_PREFIXED_EXPORTS is now provided upstream and stays set; the other fork-only build defines go, as the driver sets the socket buffer sizes itself. No user-visible change. Signed-off-by: wiznet-grace <[email protected]>
The fork called a yield hook from the blocking loops in socket.c. ioLibrary_Driver has none, so a blocked recv() could not be interrupted with Ctrl-C. Open the sockets non-blocking instead and wait in the driver, calling mp_event_handle_nowait() between attempts. Only the WIZnet TCP stack uses this code; lwIP builds are unaffected. Signed-off-by: wiznet-grace <[email protected]>
List every chip the driver now supports rather than just W5200 and W5500, and show the no-argument constructor used on boards that have the chip fitted, alongside the existing external-module example. Signed-off-by: wiznet-grace <[email protected]>
35632a7 to
4788edb
Compare
Summary
Point
lib/wiznet5kat WIZnet's ioLibrary_Driver instead of a fork that carries MicroPython-specific patches and has drifted from the library WIZnet maintains and tests, and rewriteextmod/network_wiznet5k.cagainst it. The library is vendored unchanged. WIZnet unified it last year, so one codebase now covers W5100 through W6300, which is what lets the later PRs add the W6xxx parts without a second driver.The API is unchanged: same
network.WIZNET5Kclass, sameMICROPY_PY_NETWORK_WIZNET5Kvalues, samewiznet5k_poll()/wiznet5k_deinit()entry points. No board files are touched, and the port build files only lose the fork-only defines.The rewrite also carries a few fixes the driver needed anyway:
active(False)/active(True)The W6100 and W6300 register branches are in the driver already, but nothing selects those chips until the boards PR, so that code is not reachable yet.
First of the PRs splitting up #18035, which stays open for now. Once this is merged:
machine.PIOSPIon the rp2 port, plus the W6300 and W55RP20 boards, which talk to the chip over PIO and need itsetsockopt, which the old driver never hadTesting
Hardware, on W5100S-EVB-Pico and W5500-EVB-Pico, each built both with lwIP and with the WIZnet stack (
MICROPY_PY_LWIP=0):ifconfig(),status(), DNSactive(False)/active(True)recv(), which is what the second commit is forBuilds, with no warnings and no warning suppression:
The warnings the library's own headers used to raise were fixed upstream in Wiznet/ioLibrary_Driver#186, which the submodule is pinned to.
Known gaps on the WIZnet stack, the same before and after this change, so not regressions:
socket.settimeout()is unimplementedrecv()raisesOSError(7)after the peer closes instead of returningb""DNS_run()only gives up on a one-second tick that the driver has never driven.gethostbyname()now refuses the lookup when the interface has no address, which is the way that is easy to hit by accident, but a DNS server that simply stays silent still hangs.All three need the same waiting machinery, so they go with the socket timeout PR at the end of the series.
Trade-offs and Alternatives
lib/wiznet5know points at a different repository, so existing checkouts needgit submodule sync --recursivebeforegit submodule update --init --recursive.network.WIZNET5Kkeeps its name even though the driver now covers the W6xxx parts too, so existing user code is unaffected.socket.c, which upstream has no equivalent for. Adding one to ioLibrary_Driver was the obvious fix, but that library has plenty of other users and shouldn't carry a MicroPython hook. The second commit opens the sockets non-blocking and waits in the driver instead, which the library already supports and which is also where a socket timeout has to live.