Skip to content

[Package] Exclude unused dependencies when building package.json files#3232

Draft
mho22 wants to merge 5 commits intotrunkfrom
exclude-unused-dependencies-when-building-package-json-files
Draft

[Package] Exclude unused dependencies when building package.json files#3232
mho22 wants to merge 5 commits intotrunkfrom
exclude-unused-dependencies-when-building-package-json-files

Conversation

@mho22
Copy link
Copy Markdown
Collaborator

@mho22 mho22 commented Feb 2, 2026

Motivation for the change, related issues

Based on this issue

While working on #3093, I found out @php-wasm/node was installed alongside @php-wasm/web. While looking deeper into it, I found in the package-lock.json file that fs-journal had @php-wasm/node as an unused dependency, since it was only used in fs-journal test files.

The root cause : NX's createPackageJson scans all project files, including test files, to resolve dependencies, and the custom getMonorepoDependencies function includes all static dependency edges from the project graph regardless of whether they come from source or test files. This caused test-only dependencies to leak into published package.json files.

Additionally, createPackageJson flattens transitive dependencies, which meant packages like @php-wasm/web ended up declaring dozens of indirect npm dependencies like octokit or async-lock that belong to their transitive dependencies, not to them directly.

Implementation details

Instead of maintaining a manual filteredDependencies whitelist per package, the package-json executor now automatically excludes test-only dependencies by:

  1. Reading NX's file map cache and filtering out files in test/ directories.
  2. Passing the filtered file map to createPackageJson so it only resolves dependencies from source files.
  3. Building a set of direct source dependency targets and using it to filter both monorepo dependencies and external npm dependencies from the createPackageJson output.

This also required moving test files into dedicated src/test/ directories across published packages, so the directory-based filter works consistently. There are still directories that doesn't have a test directory :

  • playground-personal-wp
  • playground-website-extras
  • playground-website
  • nx-extensions

I also removed 10 unused dependency artifacts [ probably related to former isomorphic-git ] from @wp-playground/storage's package.json.

Testing Instructions (or ideally a Blueprint)

CI

🧪 test-built-npm-packages/commonjs-and-jest/tests/deps.spec.ts
🧪 test-built-npm-packages/es-modules-and-vitest/tests/deps.spec.ts

or manually run :

In trunk : node_modules/.bin/nx reset && node_modules/.bin/nx run php-wasm-web:build

and compare the dist/packages/php-wasm/web/package.json with the one built in this branch.

@mho22 mho22 added [Type] Bug An existing feature does not function as intended [Feature] PHP.wasm dependencies Pull requests that update a dependency file labels Feb 2, 2026
@mho22 mho22 marked this pull request as draft February 2, 2026 12:38
@mho22
Copy link
Copy Markdown
Collaborator Author

mho22 commented Feb 2, 2026

@adamziel @php-wasm/fs-journal package.json file isn't listing the @php-wasm/node package anymore but it still lists this :

  "dependencies": {
    "express": "4.22.0",
    "ini": "4.1.2",
    "wasm-feature-detect": "1.8.0",
    "ws": "8.18.3",
    "yargs": "17.7.2",
    "@php-wasm/universal": "3.0.46",
    "@php-wasm/util": "3.0.46",
    "@php-wasm/logger": "3.0.46"
  },

I can reliably say wasm-feature-detect comes from @php-wasm/node-builds here. So even if I removed the @php-wasm/node dependency from the package.json file, it still adds the dependencies that are needed inside excluded dependencies.

That is expected since I only indicate which dependency should be ignored. Two solutions :

- "excludedDependencies": ["@php-wasm/node"]
+ "excludedDependencies": ["@php-wasm/node", "express", "ini", "wasm-feature-detect", "ws", "yargs"]

Or we should find another way to ignore these subdependencies.

Maybe this :

packages/nx-extensions/src/executors/package-json/executor.ts on line 139 :

- if (!packageJson.dependencies) {
        packageJson.dependencies = {};
- }

for (const dep of monorepoDependencies) {
	packageJson.dependencies[dep.name] = dep.version;
}

I expect having to clean each package.json file from their unused dependencies.

@mho22
Copy link
Copy Markdown
Collaborator Author

mho22 commented Feb 2, 2026

I may have a better suggestion here. Instead of excludedDependecies we could have filteredDependencies which lists the dependencies we actually need.

For example @php-wasm/fs-journal needs :

  "dependencies": {
-    "express": "4.22.0",
-    "ini": "4.1.2",
-    "wasm-feature-detect": "1.8.0",
-    "ws": "8.18.3",
-    "yargs": "17.7.2",
    "@php-wasm/universal": "3.0.46",
    "@php-wasm/util": "3.0.46",
    "@php-wasm/logger": "3.0.46"
-    "@php-wasm/node": "3.0.46"
  },

So filteredDependencies could be:

"filteredDependencies": ["@php-wasm/universal", "@php-wasm/util", "@php-wasm/logger"]

Same with @php-wasm/node-builds/8-5 which needs :

  "dependencies": {
-  "ini": "4.1.2",
    "wasm-feature-detect": "1.8.0",
-  "ws": "8.18.3",
    "@php-wasm/universal": "3.0.46"
  },

So :

"filteredDependencies": ["@php-wasm/universal", "wasm-feature-detect"]

ini is used in @php-wasm/universal while ws from, well, I don't know yet.

@mho22 mho22 force-pushed the exclude-unused-dependencies-when-building-package-json-files branch from 6a75247 to 3b16f72 Compare April 7, 2026 11:39
@mho22 mho22 force-pushed the exclude-unused-dependencies-when-building-package-json-files branch from 3b16f72 to 55d3597 Compare April 8, 2026 15:02
@mho22 mho22 force-pushed the exclude-unused-dependencies-when-building-package-json-files branch from 55d3597 to e629587 Compare April 8, 2026 15:19
@mho22 mho22 marked this pull request as ready for review April 8, 2026 15:44
@mho22 mho22 requested review from a team, bgrgicak and Copilot April 8, 2026 15:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR updates the Nx package-json executor to prevent test-only and transitive dependencies from leaking into published package.json files, aligning declared dependencies with actual source imports.

Changes:

  • Filter Nx file map to exclude test directories and use it to drive createPackageJson dependency detection.
  • Remove non-source (transitive / test-derived) dependencies from the generated package.json output.
  • Rewire many tests to import from ../lib/... and add dependency-verification tests for built npm packages.

Reviewed changes

Copilot reviewed 61 out of 78 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
packages/playground/wordpress-builds/src/test/get-wordpress-module-details.spec.ts Updates test import path to match new source vs test folder layout.
packages/playground/test-built-npm-packages/es-modules-and-vitest/tests/deps.spec.ts Adds an automated check that built packages only declare deps used by source imports.
packages/playground/test-built-npm-packages/es-modules-and-vitest/run-tests.ts Runs both dependency checks and existing tests per PHP version.
packages/playground/test-built-npm-packages/commonjs-and-jest/tests/deps.spec.ts Adds the same dependency declaration check for the CJS/Jest build pipeline.
packages/playground/storage/src/test/paths.spec.ts Updates test imports to reference ../lib modules.
packages/playground/storage/src/test/git-sparse-checkout.spec.ts Updates test imports to reference ../lib modules.
packages/playground/storage/src/test/filesystems.spec.ts Updates imports/types and fixture path to align with new test folder structure.
packages/playground/storage/src/test/changeset.spec.ts Updates test import to reference ../lib.
packages/playground/storage/package.json Removes dependency artifacts so the package only declares needed runtime deps.
packages/playground/blueprints/src/tests/v1/resources.spec.ts Updates test import paths to reference lib output.
packages/playground/blueprints/src/tests/v1/compile.spec.ts Updates imports to lib and adjusts fixture path resolution.
packages/playground/blueprints/src/tests/steps/write-files.spec.ts Updates test import path to reference lib step implementation.
packages/playground/blueprints/src/tests/steps/write-file.spec.ts Updates test import path to reference lib step implementation.
packages/playground/blueprints/src/tests/steps/wp-cli.spec.ts Updates imports to lib and adjusts fixture path resolution.
packages/playground/blueprints/src/tests/steps/site-data.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/set-site-language.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/run-sql.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/run-php.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/rmdir.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/rm.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/reset-data.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/mv.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/mkdir.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/login.spec.ts Updates step imports to reference lib.
packages/playground/blueprints/src/tests/steps/install-theme.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/install-plugin.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/import-wxr.spec.ts Updates imports to lib and adjusts fixture paths.
packages/playground/blueprints/src/tests/steps/import-wordpress-files.spec.ts Updates step imports to reference lib.
packages/playground/blueprints/src/tests/steps/import-theme-starter-content.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/enable-multisite.spec.ts Updates step imports and adjusts fixture paths.
packages/playground/blueprints/src/tests/steps/define-wp-config-consts.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/cp.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/activate-theme.spec.ts Updates step import path to reference lib.
packages/playground/blueprints/src/tests/steps/activate-plugin.spec.ts Updates step import path to reference lib.
packages/php-wasm/web/src/test/tls/prf.spec.ts Updates test import path to reference lib TLS implementation.
packages/php-wasm/web/src/test/tls/certificates.spec.ts Updates TLS test imports to reference lib.
packages/php-wasm/web/src/test/tcp-over-fetch-websocket.spec.ts Updates imports to lib and adjusts test util import path.
packages/php-wasm/web/playwright.config.ts Restricts Playwright matching to a specific spec file.
packages/php-wasm/web-service-worker/src/test/utils.spec.ts Updates test imports to lib and adds explicit Vitest imports.
packages/php-wasm/web-service-worker/src/test/fetch-with-cors-proxy.spec.ts Updates test imports to lib.
packages/php-wasm/web-service-worker/src/index.ts Changes package exports to re-export from ./lib/*.
packages/php-wasm/util/src/test/split-shell-command.spec.ts Updates test import path to reference ../lib.
packages/php-wasm/util/src/test/semaphore.spec.ts Updates test import path to reference ../lib.
packages/php-wasm/util/src/test/paths.spec.ts Updates test import path to reference ../lib.
packages/php-wasm/util/src/test/create-spawn-handler.spec.ts Updates test imports to reference ../lib.
packages/php-wasm/universal/src/test/process-id-allocator.spec.ts Updates test import path to reference ../lib.
packages/php-wasm/universal/src/test/php-worker.spec.ts Updates test imports to reference ../lib.
packages/php-wasm/universal/src/test/php-response.spec.ts Updates test import path to reference ../lib.
packages/php-wasm/universal/src/test/php-request-handler.spec.ts Updates test imports to reference ../lib.
packages/php-wasm/universal/src/test/object-pool-proxy.spec.ts Updates test import path to reference ../lib.
packages/php-wasm/universal/src/test/http-cookie-store.spec.ts Updates test import path to reference ../lib.
packages/php-wasm/universal/src/test/file-lock-manager-in-memory.spec.ts Updates test import path to reference ../lib.
packages/php-wasm/universal/src/test/file-lock-interval-tree.spec.ts Updates test imports to reference ../lib.
packages/php-wasm/universal/src/test/comlink-sync.spec.ts Updates test import path and adjusts referenced file path to ../lib.
packages/php-wasm/scopes/src/test/scope.test.ts Updates tests to import from a new ../lib/scope module.
packages/php-wasm/scopes/src/lib/scope.ts Extracts scope URL helpers into src/lib for cleaner source/test separation.
packages/php-wasm/scopes/src/index.ts Re-exports from ./lib/scope instead of defining logic inline.
packages/php-wasm/progress/src/test/progress-tracker.spec.ts Updates test imports to reference ../lib.
packages/php-wasm/node-polyfills/src/test/custom-event.spec.ts Updates test import to reference ../lib.
packages/php-wasm/node-polyfills/src/test/blob.spec.ts Updates test import to reference ../lib.
packages/nx-extensions/src/executors/package-json/executor.ts Implements source-only dependency detection and filters generated dependency lists accordingly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +226 to +236
function getSourceOnlyFileMap(): ProjectFileMap {
const cache = readFileMapCache();
const fullFileMap = cache?.fileMap?.projectFileMap || {};
const filtered: ProjectFileMap = {};
for (const [project, files] of Object.entries(fullFileMap)) {
filtered[project] = (files as FileData[]).filter((f) =>
isSourceFile(f.file)
);
}
return filtered;
}
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the Nx file map cache is missing/unavailable, fullFileMap becomes {}, and the executor still passes the resulting empty map into createPackageJson and uses it to compute sourceDeps. This can cause all dependencies (including legitimate runtime deps) to be dropped from the generated package.json. Consider returning undefined/null when the cache is unavailable and skipping both createPackageJson(..., sourceFileMap) and dependency filtering in that case (or falling back to the unfiltered createPackageJson behavior).

Copilot uses AI. Check for mistakes.
Comment on lines +20 to +21
import { readFileMapCache } from 'nx/src/project-graph/nx-deps-cache';
import { fileDataDepTarget } from 'nx/src/config/project-graph';
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These imports reach into Nx internal source paths (nx/src/...), which are not part of the public API surface and can break on Nx upgrades or even patch releases. If possible, prefer a public @nx/devkit API (or encapsulate this behind a small compatibility layer with a well-defined fallback) so the executor remains stable across Nx versions.

Copilot uses AI. Check for mistakes.
Comment on lines +33 to +37
if (entry.isDirectory()) {
if (entry.name !== 'test' && entry.name !== 'node_modules') {
walk(full);
}
} else if (
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This dependency check intends to ignore test code, but it only skips a directory named test. The repo also contains tests under src/tests (plural), so those files will still be scanned and their imports will be treated as “source imports”, weakening the check and potentially masking dependency leaks. Update the walker to exclude both test and tests directories (and keep the logic aligned with the executor’s test-file filtering).

Copilot uses AI. Check for mistakes.
@mho22 mho22 marked this pull request as draft April 8, 2026 15:53
@mho22 mho22 removed the request for review from bgrgicak April 8, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file [Type] Bug An existing feature does not function as intended

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants