Skip to content

unfs3 probably broken #127

Description

@sjorge

Looks like unfs3 package is broken on Solaris

2018-07-12T21:51:11+02:00 apollo unfsd[925368]: [ID 952037 user.error] svc_tli_create: could not do t_sync: Illegal file descriptor

Is all that is logged and it does not appear to start.

Activity

  1. jclulow commented on Jul 12, 2018

    @jclulow

    This is an unfortunately complicated problem.

    If you attempt to use the standard NFS port (2049), unfs3 will create a regular BSD style TCP socket and bind it to the specified port. It will then pass that file descriptor to svctcp_create(), part of the RPC suite of functions. On Solaris and illumos systems, that function comes from libnsl.so.1 and appears to require not a BSD socket but a TLI socket, an unfortunate remnant of UNIX history.

    Note in the autoconf scripts for unfs3 there is a mention of librpcsoc.so.1, one of the SunOS/UCB compatibility libraries that we don't ship in SmartOS. This library contains an implementation of svctcp_create() which does allow the use of BSD sockets. I believe that the configure script doesn't end up using it for our pkgsrc build because it's not available on the system, but the behaviour you see above is then only visible at runtime.

    One way to work around this, though it's not perfect (because it involves the use of dynamic port numbers which confuses at least the Linux NFS client) is to use the -u flag to unfsd. This will use a random port for the NFS and MOUNT services; i.e., it passes RPC_SOCKANY rather than a specific file descriptor, so there is no socket type mismatch.

    There are two likely paths forward for this issue:

    1. fix illumos to accept BSD sockets as well as TLI sockets for the RPC functions, though this means unfsd will still be broken until you upgrade to a platform image with this fix
    2. fix unfsd to be able to create and bind a TLI socket instead of a BSD socket, which would mean we can use unfsd even on older illumos systems in advance of this fix

    There's a third option, which is probably also not great, which is to create a pkgsrc package with the librpcsoc.so.1 compatibility bits in it and ship it outside the core OS -- but I don't know how hard it would be to extricate it from illumos-gate and build it elsewhere, or if it makes use of private interfaces with the rest of the OS.

  2. jclulow commented on Jul 12, 2018

    @jclulow

    Note in particular that the compatibility librpcsoc.so.1 contains what appear to be wrappers around the core RPC functionality from libnsl.so.1; e.g., see https://github.com/joyent/illumos-joyent/blob/master/usr/src/ucblib/librpcsoc/svc_tcp.c#L87-L168

    This function calls at least svc_xprt_alloc() which does not seem to be documented, and xprt_register(), which does. Some more digging would be required to see how safe it would be to ship a copy of this library in pkgsrc.

  3. sjorge commented on Jul 13, 2018

    @sjorge
    Author

    I actually tried specifying a udp port but that did not help. This is probably related why some packages no longer build without patching because they can't find rpcgen on newer platforms. (See #126) Maybe it is worth wrapping some of the remove bits up in an optional package somehow.

  4. jclulow commented on Jul 13, 2018

    @jclulow

    @sjorge Specifying a port won't help, because from the perspective of the RPC routines the same thing is happening. Whether unfsd attempts to use its specific default (2049), or whether you provide a specific port, in both cases unfsd is binding a BSD style socket and attempting to pass it to the RPC routines, which doesn't work for the reason I described above.

    If instead you use the -u flag I mentioned, the software will pass RPC_SOCKANY to get a random port number. From the manual page:

           -u     Use an unprivileged port for NFS and MOUNT service. Normally,
                  unfsd will use port number 2049, which is the standard port for
                  NFS.  When this option is in effect, arbitrary ports chosen by
                  the RPC library will be used. You may need to use this option
                  when running unfsd from a normal user account.
    

    This is not related to rpcgen -- rather, it's a long-standing issue with unfsd when built against the regular libnsl.so.1 RPC routines. As far as I know, we've never shipped the ancient compatibility librpcsoc.so.1 in the SmartOS platform image so this problem has existed in the unfsd package since we started shipping pkgsrc packages.

  5. added a commit that references this issue on Aug 20, 2018
  6. added a commit that references this issue on Sep 24, 2018
  7. added a commit that references this issue on Jan 10, 2019
  8. added a commit that references this issue on Sep 2, 2019
  9. added a commit that references this issue on Oct 20, 2019
  10. added a commit that references this issue on Mar 11, 2020
  11. 94 remaining items

  12. added a commit that references this issue on Aug 26, 2026
  13. added a commit that references this issue on Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions