Skip to content

Bootstrapping fails on OpenMPTCProuter v0.59 (a build of OpenWRT) #130

Open
Open
Bootstrapping fails on OpenMPTCProuter v0.59 (a build of OpenWRT)#130

Description

@sskras
sskras
opened on Mar 14, 2024

To run the bootstrap I needed to fake groups and whoami executables. The related (sub-) issue: #137

Then it failed due to missing nawk:

root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# bootstrap/bootstrap
  ...
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.
com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/git
hub.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/g
ithub.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/githu
b.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: {print $3; exit}: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/tools/gmake.mk" line 43: warning: "/usr/bin/make --version |                           '{print $3; exit}'" returned non-zero status
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECON
F=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: {print $3; exit}: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/tools/gmake.mk" line 43: warning: "/usr/bin/make --version |                           '{print $3; exit}'" returned non-zero status
/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/pkgformat/pkg/list-dependencies: line 47: /usr/pkg/bin/nawk: not found
===> Installing dependencies for cwrappers-20220403
/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/pkgformat/pkg/list-dependencies: line 47: /usr/pkg/bin/nawk: not found
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

The OS:

root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# uname -a
Linux OpenMPTCProuter 5.4.194 #0 SMP Tue May 17 22:11:28 2022 x86_64 GNU/Linux

root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# cat /etc/os-release
NAME="openmptcprouter"
VERSION="v0.59.1-5.4"
ID="openmptcprouter"
ID_LIKE="lede openwrt"
PRETTY_NAME="openmptcprouter v0.59.1-5.4"
VERSION_ID="v0.59.1-5.4"
HOME_URL="https://openwrt.org/"
BUG_URL="https://bugs.openwrt.org/"
SUPPORT_URL="https://forum.openwrt.org/"
BUILD_ID="r0+16594-ce92de8c8c"
OPENWRT_BOARD="x86/64"
OPENWRT_ARCH="x86_64"
OPENWRT_TAINTS="busybox override"
OPENWRT_DEVICE_MANUFACTURER="OpenWrt"
OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/"
OPENWRT_DEVICE_PRODUCT="Generic"
OPENWRT_DEVICE_REVISION="v0"
OPENWRT_RELEASE="openmptcprouter v0.59.1-5.4 r0+16594-ce92de8c8c"

Activity

  1. dhgutteridge commented on Mar 22, 2024

    @dhgutteridge
    dhgutteridge
    on Mar 22, 2024
    Member

    I don't have an OpenMPTCProuter or OpenWrt environment to test with,
    but searches suggest there is a BusyBox non-GNU implementation of awk
    included. That wouldn't get picked up with the present assumptions that
    some pkgsrc tooling makes. I don't know if that BusyBox version would
    be adequate anyway, as various others don't meet pkgsrc expectations.
    It's probably simplest to force use of nawk here. Please try this
    patch.

    --- bootstrap.orig
+++ bootstrap
@@ -743,6 +743,8 @@
 	elif grep -sq '^CHROMEOS_RELEASE_NAME' /etc/lsb-release; then
 		need_awk=yes
 		need_sed=yes
+	elif grep -sq openwrt /etc/os-release; then
+		need_awk=yes
 	fi
 	machine_arch=`uname -m`
 	# Override machine_arch where required.

    (Now, I'm also not sure if it's best to just handle OpenWrt and
    derivates, or search for a BusyBox indicator instead, etc., but this
    should be a starting point. According to BusyBox documentation, their
    grep should handle the options I applied.)

    You may also find the BusyBox sed implementation is inadequate (though,
    from browsing the documentation, it seems it may be fine), in which
    case you'd also need to add another line with "need_sed=yes" as well.
    (Depending on when you last pulled from GitHub, you'd also need to
    update textproc/nbsed first, as it may not build for you without a fix
    I applied on the 20th.)

    Dave

  2. jwbowen commented on Mar 24, 2024

    @jwbowen
    jwbowen
    on Mar 24, 2024 via email
    Sponsor

  3. sskras commented on Mar 24, 2024

    @sskras
    sskras
    on Mar 24, 2024
    Author

    @dhgutteridge commented 2 days ago:

    searches suggest there is a BusyBox non-GNU implementation of awk included.

    True:

    root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# command -v gawk
root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# command -v nawk
root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# command -v mawk
root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# command -v awk
/usr/bin/awk

    That wouldn't get picked up with the present assumptions that some pkgsrc tooling makes.

    Can you be more specific, please? I tried looking into mk/tools/tools.Linux.mk:

    root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# grep -iC1 awk mk/tools/tools.Linux.mk
.  endif
.  if exists(${_path}/gawk)
TOOLS_PLATFORM.gawk?=           ${_path}/gawk
.  endif
--

# assuming grep/sed/tar/awk are usually the GNU versions, is this safe?

--

TOOLS_PLATFORM.awk?=            ${TOOLS_PLATFORM.gawk}

    ... but I don't understand a thing yet. :)

    I don't know if that BusyBox version would be adequate anyway, as various others don't meet pkgsrc expectations.

    The BusyBox version seems to be a mix of features of nawk, gawk and whatnot:
    https://wiki.alpinelinux.org/wiki/Awk

    It's probably simplest to force use of nawk here. Please try this patch.

    Thanks, but before that I would like to understand the picking algorithm. Especially:

    1. Where does the expectation that /usr/pkg/bin/nawk will be present come from?
    2. Why is AWK variable left uninitialized?
      (Which leads to several sh: -f: not found errors near the end)
    --- bootstrap.orig
+++ bootstrap
@@ -743,6 +743,8 @@
 	elif grep -sq '^CHROMEOS_RELEASE_NAME' /etc/lsb-release; then
 		need_awk=yes
 		need_sed=yes
+	elif grep -sq openwrt /etc/os-release; then
+		need_awk=yes
 	fi
 	machine_arch=`uname -m`
 	# Override machine_arch where required.

    (Now, I'm also not sure if it's best to just handle OpenWrt and derivates, or search for a BusyBox indicator instead, etc., but this should be a starting point. According to BusyBox documentation, their grep should handle the options I applied.)

    BTW, grep handles this fine.

    Tried going with -A d for bmake:

    --- a/bootstrap/bootstrap
+++ b/bootstrap/bootstrap
@@ -1471,13 +1471,13 @@ build_package_nopreserve() {
 # Special packages that we don't want marked with BOOTSTRAP_PKG, but must be
 # built (if required) without -DPKG_PRESERVE set so that they can be deleted.
 #
-use_cwrappers=`(cd $pkgsrcdir/devel/bmake && $bmake show-var VARNAME=_USE_CWRAPPERS)`
+use_cwrappers=`(cd $pkgsrcdir/devel/bmake && $bmake -d A show-var VARNAME=_USE_CWRAPPERS)`
 case "$use_cwrappers" in
 yes)
        build_package_nopreserve "pkgtools/cwrappers"
        ;;
 esac
-use_mktools=`(cd $pkgsrcdir/devel/bmake && $bmake show-var VARNAME=_PKGSRC_USE_MKTOOLS)`
+use_mktools=`(cd $pkgsrcdir/devel/bmake && $bmake -d A show-var VARNAME=_PKGSRC_USE_MKTOOLS)`
 case "$use_mktools" in
 yes)
        build_package_nopreserve "pkgtools/mktools"

    ... but that left me overwhelmed with the output (which increased from 79k to 30M in size).

    Any ideas on how to proceed with finding the root causes?

  4. dhgutteridge commented on Mar 25, 2024

    @dhgutteridge
    dhgutteridge
    on Mar 25, 2024
    Member

    Sorry, I think we're coming with different ideas of what was intended. My assumption was your primary interest was bootstrapping and using pkgsrc ASAP. It's probable you will encounter further issues as you go, so it could turn into a more protracted process to walk through every question that arises. The simple answer without getting into a lot of details is that "stuff will break" if bootstrapping goes awry. You may find make components behave inconsistently, as you see here (${AWK} evaluating to nothing, or to an unexpected default). This is code written by different people at different times.

    Yes, some of this could be handled better; I'm not arguing otherwise. If you'd like to have a detailed technical review, that's a perfectly fine thing to request, but it's not something I can prioritize, I'm afraid.

  5. sskras commented on Mar 25, 2024

    @sskras
    sskras
    on Mar 25, 2024
    Author

    @dhgutteridge, thanks for making things clear!

    It's probable you will encounter further issues as you go, so it could turn into a more protracted process to walk through every question that arises.

    I thought about progressing ASAP at first. But the delay in responses made me to consider doing an RCA. Which I think is still a good thing to do in most cases.

    The simple answer without getting into a lot of details is that "stuff will break" if bootstrapping goes awry.

    [...]

    If you'd like to have a detailed technical review, that's a perfectly fine thing to request, but it's not something I can prioritize, I'm afraid.

    Fair enough. I am willing to do the research myself, but I need some (at least initial) guidance. It's hard to notice the moment where bootstrapping starts missing something and diverges from the plan.

    I'll file a new issue to walk through that in parallel. Or maybe discussing it online like on IRC would be better? I already started a talk on that with @jperkin.

    Now back to testing your fix.

  6. sskras commented on Mar 25, 2024

    @sskras
    sskras
    on Mar 25, 2024
    Author

    I tried to take the hint from @jwbowen into account:

    --- a/bootstrap/bootstrap
+++ b/bootstrap/bootstrap
@@ -748,6 +748,9 @@ Linux)
        elif grep -sq '^CHROMEOS_RELEASE_NAME' /etc/lsb-release; then
                need_awk=yes
                need_sed=yes
+       # Busybox distros have some things broken.
+       elif command -v awk | xargs ls -l | grep -sq busybox; then
+               need_awk=yes
        fi
        machine_arch=`uname -m`
        # Override machine_arch where required.

    ... and the error changed:

    ===> bootstrap command: bootstrap/bootstrap 
===> bootstrap started: Mon Mar 25 13:00:12 UTC 2024
===> running: /bin/sed -e 's|@DEFAULT_INSTALL_MODE@|'0755'|' /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/install-sh/files/install-sh.in > /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh
===> running: /bin/chmod +x /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh
===> Creating default mk.conf in /root/debug/src/github.com/NetBSD/pkgsrc.git/work
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk
===> Bootstrapping mk-files
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/bootstrap-mk-files/files && env CP=/bin/cp  OPSYS=Linux MK_DST=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk ROOT_GROUP=root ROOT_USER=root SED=/bin/sed SYSCONFDIR=/usr/pkg/etc /bin/sh ./bootstrap.sh)
===> Bootstrapping bmake
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /bin/sh configure  --prefix=/root/debug/src/github.com/NetBSD/pkgsrc.git/work --with-default-sys-path=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk --with-machine-arch=x86_64 )
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /bin/sh make-bootstrap.sh)
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake/bmake /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake
===> Building libnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat; /bin/sh ./configure  -C --prefix=/usr/pkg --infodir=/usr/pkg/info --mandir=/usr/pkg/man --sysconfdir=/usr/pkg/etc --enable-bsd-getopt --enable-db && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1)
===> Bootstrapping awk
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1 -f Makefile CC="cc" CFLAGS="")
cc   -c awkgram.tab.c
sh: cc: not found
*** [awkgram.tab.o] Error code 127

bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk
1 error

bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk
===> exited with status 2
aborted.

    Now it misses cc, although the above steps like Bootstrapping bmake and Building libnbcompat dealt with no cc just fine.

    Is having a cc or ${CC} defined a requirement of pkgsrc? Maybe just another, minor bug?

  7. sskras commented on Mar 25, 2024

    @sskras
    sskras
    on Mar 25, 2024
    Author

    Now after defining CC=gcc pkgsrc gets into some circular dependency:

    ===> Bootstrapping pkgtools
  ...
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_admin check.o main.o -linstall  -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c pl.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c util.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c build.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_create main.o perform.o pl.o util.o build.o -linstall /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/.libs/libarchive.a   -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c show.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_info main.o perform.o show.o -linstall  -lnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECONF=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
===> Installing dependencies for cwrappers-20220403
=> Tool dependency coreutils>=5.2.1: NOT found
=> Verifying reinstall for ../../sysutils/coreutils
=> Bootstrap dependency digest>=20211023: NOT found
=> Verifying reinstall for ../../pkgtools/digest
===> Installing dependencies for digest-20220214
=> Tool dependency mktools-[0-9]*: NOT found
=> Verifying reinstall for ../../pkgtools/mktools
===> Installing dependencies for mktools-20220614
=> Tool dependency cwrappers>=20150314: NOT found
=> Verifying reinstall for ../../pkgtools/cwrappers
ERROR: This package has set PKG_FAIL_REASON:
ERROR: Circular dependency detected
*** Error code 1

Stop.
bmake[4]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
*** Error code 1

Stop.
bmake[3]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/mktools
*** Error code 1

Stop.
bmake[2]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/digest
*** Error code 1

Stop.
bmake[1]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/coreutils
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

  8. dhgutteridge commented on Mar 27, 2024

    @dhgutteridge
    dhgutteridge
    on Mar 27, 2024
    Member

    Now it misses cc, although the above steps like Bootstrapping bmake and Building libnbcompat dealt with no cc just fine.

    Is having a cc or ${CC} defined a requirement of pkgsrc? Maybe just another, minor bug?

    You haven't shared a complete log, so it's hard to know what's going on in entirety (e.g., with the other two packages that already built). This implies the individual packages have different expectations, but I haven't walked through them.

  9. dhgutteridge commented on Mar 27, 2024

    @dhgutteridge
    dhgutteridge
    on Mar 27, 2024
    Member

    Now after defining CC=gcc pkgsrc gets into some circular dependency:

    ===> Bootstrapping pkgtools
  ...
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_admin check.o main.o -linstall  -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c pl.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c util.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c build.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_create main.o perform.o pl.o util.o build.o -linstall /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/.libs/libarchive.a   -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c show.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_info main.o perform.o show.o -linstall  -lnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECONF=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
===> Installing dependencies for cwrappers-20220403
=> Tool dependency coreutils>=5.2.1: NOT found
=> Verifying reinstall for ../../sysutils/coreutils
=> Bootstrap dependency digest>=20211023: NOT found
=> Verifying reinstall for ../../pkgtools/digest
===> Installing dependencies for digest-20220214
=> Tool dependency mktools-[0-9]*: NOT found
=> Verifying reinstall for ../../pkgtools/mktools
===> Installing dependencies for mktools-20220614
=> Tool dependency cwrappers>=20150314: NOT found
=> Verifying reinstall for ../../pkgtools/cwrappers
ERROR: This package has set PKG_FAIL_REASON:
ERROR: Circular dependency detected
*** Error code 1

Stop.
bmake[4]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
*** Error code 1

Stop.
bmake[3]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/mktools
*** Error code 1

Stop.
bmake[2]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/digest
*** Error code 1

Stop.
bmake[1]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/coreutils
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

    This is a different issue entirely, and one I'm pretty sure isn't specific to your platform/setup. I think it could be reproducible elsewhere. I will look into that aspect.

  10. dhgutteridge commented on Mar 27, 2024

    @dhgutteridge
    dhgutteridge
    on Mar 27, 2024
    Member

    Now after defining CC=gcc pkgsrc gets into some circular dependency:

    ===> Bootstrapping pkgtools
  ...
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_admin check.o main.o -linstall  -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c pl.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c util.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c build.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_create main.o perform.o pl.o util.o build.o -linstall /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/.libs/libarchive.a   -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c show.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_info main.o perform.o show.o -linstall  -lnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECONF=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
===> Installing dependencies for cwrappers-20220403
=> Tool dependency coreutils>=5.2.1: NOT found
=> Verifying reinstall for ../../sysutils/coreutils
=> Bootstrap dependency digest>=20211023: NOT found
=> Verifying reinstall for ../../pkgtools/digest
===> Installing dependencies for digest-20220214
=> Tool dependency mktools-[0-9]*: NOT found
=> Verifying reinstall for ../../pkgtools/mktools
===> Installing dependencies for mktools-20220614
=> Tool dependency cwrappers>=20150314: NOT found
=> Verifying reinstall for ../../pkgtools/cwrappers
ERROR: This package has set PKG_FAIL_REASON:
ERROR: Circular dependency detected
*** Error code 1

Stop.
bmake[4]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
*** Error code 1

Stop.
bmake[3]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/mktools
*** Error code 1

Stop.
bmake[2]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/digest
*** Error code 1

Stop.
bmake[1]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/coreutils
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

    This could also relate to #124.

  11. dhgutteridge commented on Apr 12, 2024

    @dhgutteridge
    dhgutteridge
    on Apr 12, 2024
    Member

    Was the solution in #124 any help to you?

    I wasn't able to reproduce this. I thought a relatively recent change might have created a condition where this could occur (on NetBSD), but it seems not. I'm familiar with situations where this pattern can occur, but I doubt they are relevant to you. (E.g., if someone confuses overriding WRKDIR when they really want WRKOBJDIR, they would get this circular dependency.)

  12. sskras commented on Apr 12, 2024

    @sskras
    sskras
    on Apr 12, 2024
    Author

    IIRC it helped. I wanted to file a related bug about pkgsrc not indicating that hostname is missing (and trying to build coreutils instead). IMO it's fair check to do.

    Is that (detection) doable – what do you think, David?

    PS. Where did you try reproducing it BTW?

  13. dhgutteridge commented on Apr 12, 2024

    @dhgutteridge
    dhgutteridge
    on Apr 12, 2024
    Member

    IIRC it helped. I wanted to file a related bug about pkgsrc not indicating that hostname is missing (and trying to build coreutils instead). IMO it's fair check to do.

    Is that (detection) doable – what do you think, David?

    Detection is certainly possible. I'm not sure that's the best solution, would need understand it better. Evidently some Linux distros don't include hostname by default. Hard to support N number of Linux distros, each with their own ideas of things. (Doesn't scale well with the small number of pkgsrc developers.) "uname -n" would perhaps be a better choice (i.e. patch the problem software instead of introducing another dependency), but there may be conflicting historical reasons why that wasn't done.

    In general, IMO it's not realistic to expect pkgsrc to just work out of the box on any Linux distro without installing certain things in the base system (as opposed to requiring them to build during a bootstrap). Some of this is documented in README.Linux.

    PS. Where did you try reproducing it BTW?

    NetBSD 9.3_STABLE, which isn't a true "bootstrap" at all (under typical usage), but will pull in that dependency chain when someone starts from scratch. Thought I'd encountered a cyclical problem there back in February, but couldn't reproduce it now.

  14. sskras
    changed the title [-]Bootstrapping misses nawk on OpenMPTCProuter v0.59 (a build of OpenWRT)[/-] [+]Bootstrapping fails on OpenMPTCProuter v0.59 (a build of OpenWRT)[/+] on Apr 19, 2024
  15. netbsd-srcmastr
    added a commit that references this issue on Apr 29, 2024

    textproc/hs-commonmark: Update to 0.2.6

    4b31e0c

  16. 65 remaining items

  17. jakllsch
    added 6 commits that reference this issue on May 15, 2026

    www/ruby-uri: update to 1.1.1

    c7a95bf

    libsidplayfp: update to 2.16.0.

    315a4c4

    py-icalendar: update to 7.0.0.

    5654581

    devel/serie: update to 0.6.1

    9c09d26

    devel/mujs: update to mujs-1.3.9

    3963783

    devel/ruby-curses: update to 1.6.0

    cfe9777
  18. netbsd-srcmastr
    added 4 commits that reference this issue on May 21, 2026

    filesystems/fuse-bindfs: update to 1.18.4

    7997a87

    p5-YAML-Syck: update to 1.45.

    245bc8e

    devel/ruby-marcel: update to 1.2.1

    8884479

    p5-XML-LibXML: update to 2.0213.

    8702bc5
  19. netbsd-srcmastr
    added a commit that references this issue on Jun 12, 2026

    finance/rex: update to 0.2.7

    eade81b
  20. netbsd-srcmastr
    added a commit that references this issue on Jul 14, 2026

    net/xfr: update to 0.9.21

    0d83d61
  21. netbsd-srcmastr
    added a commit that references this issue on Jul 30, 2026

    Update to 0.41

    1e70527
  22. netbsd-srcmastr
    added 2 commits that reference this issue on Aug 26, 2026

    net/ttl: update to 0.22.0

    6ba23b5

    pdsh: Update to 2.36

    561dffc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions