[BUG] SBOM generation for CycloneDX generates duplicate dependencies #6967

Activity

1. jamietanna

   added

   Bugthing that needs fixingthing that needs fixing

   Needs Triageneeds review for next stepsneeds review for next steps

   Release 10.x

   on Nov 5, 2023
2. ljharb

   assigned

   bdehamer

   on Nov 5, 2023
3. jkowalleck commented on Nov 6, 2023

   

   jkowalleck

   on Nov 6, 2023

   Last edited by jkowalleck

   More actions

   did you experience the same issue when generating the SBOM via official tooling https://github.com/CycloneDX/cyclonedx-node-npm ?
4. jkowalleck commented on Nov 6, 2023

   

   jkowalleck

   on Nov 6, 2023

   Last edited by jkowalleck

   More actions

   @bdehamer see my earlier remarks related to intrinsic impossible deduplication in node_modules: npm/rfcs#714 (comment)
5. bdehamer commented on Nov 30, 2023

   

   bdehamer

   on Nov 30, 2023

   Last edited by bdehamer

   Contributor

   More actions

   @jamietanna I'm digging into this issue and considering a couple different solutions. I'd be curious to hear which of these best meets the need of your SBOM use cases . . .

   The Issue

   In certain circumstances, it is not possible for npm to completely deduplicate packages in the node_modules tree. A basic example would be something like this:

   [email protected]
   ├─┬ [email protected]
   │ └── [email protected]
   ├─┬ [email protected]
   │ └── [email protected]
   └── [email protected]
   

   My demo-package project has dependencies on foo, bar and tslib (version 2.6.2). Since foo and bar each have a dependency on an older version of tslib (version 1.14.1) that is in conflict with the version needed by the root project, [email protected] cannot be hoisted to top of the node_modules and ends-up being duplicated under both foo and bar.

   Since version 1.14.1 of tslib literally appears on-disk at two different locations in the tree, the somewhat naive SBOM generator ends-up adding two identical entries to the CycloneDX components list.

   This is why the resulting SBOM fails validation -- we end up with multiple entries which have identical bom-ref values.

   Solution 1

   One way to address this would be to treat each package that appears in the tree as a distinct dependency -- even if it is technically identical to some other dependency already present in the tree.

   Given the example above, this solution would result in [email protected] being listed twice in the SBOM, albeit with a distinct bom-ref value. We might choose to do something like prefix the bom-ref name the parent package name resulting in entries that look something like:

   [
     {
       "bom-ref": "[email protected]@1.14.1",
       "type": "library",
       "name": "tslib",
       "version": "1.14.1",
     },
     {
       "bom-ref": "[email protected]@1.14.1",
       "type": "library",
       "name": "tslib",
       "version": "1.14.1",
     }
   ]

   I believe that this is similar to the how cyclonedx-node-npm solves this problem.

   Solution 2

   The other approach would be to deduplicate that packages before adding them to the SBOM. Instead of literally mirroring the layout of packages in the node_modules directory, this solution would detect the multiple instances of [email protected] and fold them into a single entry in the SBOM components list:

   [
     {
       "bom-ref": "[email protected]",
       "type": "library",
       "name": "tslib",
       "version": "1.14.1",
     }
   ]

   In this case, we're not trying to represent the layout of the node_modules directory, but instead just enumerating the distinct dependencies that comprise the project. This is how both cdxgen and the snyk SBOM command handle the issue of duplicate packages.

   ---

   I think there are cases to be made for either of these solutions, but I'd like to know which of these best matches the output you'd expect to see in a valid SBOM?
6. savek-cc commented on Apr 26, 2024

   

   savek-cc

   on Apr 26, 2024

   More actions

   It's been a while - but I'd strongly vote for option 2.
   If a technical identical dependency is included multiple times, it should only appear ONCE as a component in the SBOM.
   It can be referenced multiple times in the dependency section of the sbom though. Compare to what maven is also doing.

   👍React with 👍5Reacted by bu-michael, Mats, Michael Chan, Johannes Loher and joseluiswork

   👎React with 👎1Reacted by Jan Kowalleck
7. ghost91- commented on Sep 16, 2024

   

   ghost91-

   on Sep 16, 2024

   More actions

   We use OWASP Dependency-Track and just updated it to v4.11.1. It now validates uploaded SBOMs and rejects those generated by npm:

   [2024-09-16T04:37:17.479Z] [DependencyTrack] {"status":400,"title":"The uploaded BOM is invalid","detail":"Schema validation failed","errors":["$.components[320].externalReferences[2].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference","$.components[320].externalReferences[2].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference","$.components[320].externalReferences[2].url: does not match the regex pattern ^urn:cdx:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/[1-9][0-9]*$","$.components[320].externalReferences[2].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference","$.components[320].externalReferences[2].url: does not match the regex pattern ^urn:cdx:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/[1-9][0-9]*#.+$","$.dependencies: the items in the array must be unique"]}
   

   It sounds like the problem is duplicate entries.

   The official tooling does not exhibit this problem.

   👍React with 👍4Reacted by bu-michael, Jan Eisenmenger, Patrick Harböck and Michael Chan
8. jkowalleck commented on Oct 24, 2024

   

   jkowalleck

   on Oct 24, 2024

   More actions

   You may want to try CycloneDX's official SBOM generator for NPM - it is properly maintained and does not have those issues.

   👍React with 👍1Reacted by Gaël Jourdan-Weil
9. wraithgar

   closed this as completedin ab9ddc0on Dec 20, 2024
10. wraithgar

    closed this as completedin #7992on Dec 20, 2024
11. atolcd-gerrit

    added a commit that references this issue on Jun 25, 2025

    Utilise cyclonedx-npm pour générer le SBOM

    ...

    5bc804f
12. gaeljw commented on Apr 1, 2026

    

    gaeljw

    on Apr 1, 2026

    More actions

    Switching to https://www.npmjs.com/package/@cyclonedx/cyclonedx-npm solved the issue for me. Feels weird that the official CLI tool is not maintained or generates an invalid SBOM :'(

    👍React with 👍3Reacted by Jan Kowalleck, Savek-CC and menno-snom