Skip to content

ports/unix: Add full uos.dupterm support. - #6080

Open
andrewleech wants to merge 2 commits into
micropython:masterfrom
andrewleech:unix_dupterm
Open

andrewleech wants to merge 2 commits into
micropython:masterfrom
andrewleech:unix_dupterm

Conversation

@andrewleech

Copy link
Copy Markdown
Contributor

The unix port currently supports a very limited uos.dupterm implementation.
It can only replace stdin and/or duplicate stdout.
When using uos.dupterm to replace slot 0 it does not return a handle to the existing (posix stdio) stream.

This PR resolves these issues, preferring to use dupterm for all stdio on unix when MICROPY_PY_OS_DUPTERM=1 is configured in the build.

With this change simple modification of the output stream is possible, for example to format logging messaging with a basic timestamp

stdio = None

class logger:
    def write(self, buf):
        global stdio        
        stdio.write(str(utime.time()))
        stdio.write(" | ")
        stdio.write(buf)

stdio = uos.dupterm(logger(), 0)

Comment thread ports/unix/mphalport.h Outdated
@@ -103,3 +103,7 @@ enum {

void mp_hal_get_mac(int idx, uint8_t buf[6]);
#endif

#if MICROPY_PY_OS_DUPTERM
void init_dupterm_stdio();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should have mp_ prefix since it is in a header file. Or even better, mp_unix_ since it is specific to the unix port.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was loath to change the name of an existing file, but I too thought it was oddly inconsistent in name.

Comment thread ports/unix/unix_mphal.c
@@ -64,8 +66,11 @@ STATIC void sighandler(int signum) {
}
#endif

int mp_interrupt_char = -1;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be nice to have a comment that explains this is the mp_interrupt_char from lib/utils/interupt_char.h

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Yes I had to add this just to allow compilation with MICROPY_PY_OS_DUPTERM enabled.

Comment thread ports/unix/unix_mphal.c Outdated
MP_STATE_VM(dupterm_objs[idx]) = save_term;
STATIC mp_uint_t unix_stdio_read(mp_obj_t self_in, void *buf, mp_uint_t size, int *errcode) {
ssize_t ret;
MP_HAL_RETRY_SYSCALL(ret, read(STDIN_FILENO, (byte *)buf, size), {});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be handling the error? Otherwise we end up returning -1 at the end of the function without setting *errorcode

Comment thread ports/unix/unix_mphal.c Outdated
STATIC mp_uint_t unix_stdio_read(mp_obj_t self_in, void *buf, mp_uint_t size, int *errcode) {
ssize_t ret;
MP_HAL_RETRY_SYSCALL(ret, read(STDIN_FILENO, (byte *)buf, size), {});
if (ret == 0) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought a return value of 0 meant that we have reached EOF rather than non-blocking. Is this a special case for if stdin is closed but other slots are still open?

Comment thread ports/unix/unix_mphal.c Outdated

STATIC mp_uint_t unix_stdio_write(mp_obj_t self_in, const void *buf, mp_uint_t size, int *errcode) {
int ret;
MP_HAL_RETRY_SYSCALL(ret, write(STDOUT_FILENO, (const byte *)buf, size), {});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here with not handling the error

@dpgeorge

Copy link
Copy Markdown
Member

On CPython I guess you can hook into stdout by redefining sys.stdout... and maybe that's how MicroPython should have done it from the start, instead of adding uos.dupterm() (although it'd still need an efficient multiplexer, eg sys.stdout = micropython.stream_multiplex(sys.stdout, machine.UART(...)), not to mention handling sys.stdin).

Anyway, the way it is now sys.stdout is before the dupterm multiplexer, so writing to sys.stdout goes to all dupterm streams (similar for reading). And what this PR does is adds raw-stdout/stdin objects, ie they do not go via dupterm. On bare-metal the equivalent would be a UART or USB_VCP object.

It just seems like there's a lot of complexity here with two different kinds of stdout/stdin objects (a dupterm one and a raw one). But I don't really see a way around that.

Would it be possible to reuse the extmod/vfs_posix_file.c code to implement the raw stdin/stdout? That code has compile-time dupterm support but maybe that dupterm support can be made optional at runtime (via a flag in the stdio object) so that the same code can be used for both sets of stdio objects?

@andrewleech

Copy link
Copy Markdown
Contributor Author

Thanks for the pointer towards extmod/vfs_posix_file.c - it already has the stdio object instantiated

const mp_obj_vfs_posix_file_t mp_sys_stdin_obj = {{&mp_type_textio}, STDIN_FILENO};
const mp_obj_vfs_posix_file_t mp_sys_stdout_obj = {{&mp_type_textio}, STDOUT_FILENO};
const mp_obj_vfs_posix_file_t mp_sys_stderr_obj = {{&mp_type_textio}, STDERR_FILENO};

I should be able to clean up this patch reusing them.

@andrewleech

andrewleech commented May 28, 2020 •

Copy link
Copy Markdown
Contributor Author

@dpgeorge there's a catch with using the mp_sys_stdin_obj etc objects above as i/o for dupterm, other things read/write to these objects - and writes to stdout/stderr are redirected to dupterm (hello recursion).

I've resolved this by adding a relatively small amount of extra logic to make a new const mp_obj_vfs_posix_file_t mp_sys_raw_stdio_obj which will automatically read/write to the correct fd and not go via dupterm - this new raw_stdio can be used for unix_mphal.c both with and without dupterm.

It feels like a bit of a tacked-on-the-side addition, but it does make for less duplication than I had previously.

Now I'm also getting a lot of travis fails, for one the minimal port doesn't have the new file handle created, presumably due to a lack of one/both of MICROPY_VFS_POSIX / MICROPY_VFS_POSIX_FILE
Some of the others are failing a number of unit tests, don't know why yet.

@github-actions

Copy link
Copy Markdown

Code size report:

Reference:  tools/ci.sh: Increase qemu_arm test run timeout. [5c00edc]
Comparison: ports/unix: Add full uos.dupterm support. [merge of 779e38b]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64: +2492 +0.291% standard[incl +140(data) +32(bss)]
      stm32:    +8 +0.002% PYBV10
      esp32:    +0 +0.000% ESP32_GENERIC
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@codecov

codecov Bot commented Mar 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.15%. Comparing base (2dc2e30) to head (66a4bc4).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6080      +/-   ##
==========================================
- Coverage   98.46%   98.15%   -0.32%     
==========================================
  Files         176      176              
  Lines       22784    22890     +106     
==========================================
+ Hits        22435    22467      +32     
- Misses        349      423      +74     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pi-anl added 2 commits March 31, 2026 07:37
…path.

The MAX(0, written) comparison where written is mp_uint_t (unsigned)
triggers a -Wtype-limits warning because unsigned >= 0 is always true.
Replace with an explicit MP_STREAM_ERROR check which is the actual
intended guard.

Signed-off-by: Andrew Leech <[email protected]>
Implements proper dupterm support for the unix port using the existing
MICROPY_PY_OS_DUPTERM_BUILTIN_STREAM infrastructure.

A lightweight mp_unix_stdio_obj stream type wraps STDIN/STDOUT and is
registered as dupterm slot 0 at init. This allows os.dupterm() to manage
the default console alongside user-provided streams (e.g. for remote
REPL over sockets).

The mp_hal_stdin_rx_chr and mp_hal_stdout_tx_strn functions are
simplified to delegate to mp_os_dupterm_rx_chr/mp_os_dupterm_tx_strn.

Also provides mp_interrupt_char for dupterm keyboard interrupt detection
on non-builtin streams, and removes the stderr->dupterm echo to avoid
double output on the unix port where stderr and stdout are separate.

Signed-off-by: Andrew Leech <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants