esmodule.com logo with the letters ES in capital letters in white against an orange background and the word modul next to it in white on a transparent background.

Introducing the ES Module Project

It's time to deprecate CommonJS. Introducing the ES Module Project, an independent project with the goal of driving the adoption of ES Modules in the Node.js ecosystem.
11 min read
esmodules

It's time to deprecate CommonJS. Introducing the ES Module Project, an independent project with the goal of driving the adoption of ES Modules in the Node.js ecosystem.

EcmaScript modules (ESM) were introduced into the EcmaScript standard in 2015. Node.js shipped its first LTS distribution with support for ES Modules in 2020 with Node.js v16. Since then adoption has been poor, with real world usage in Node.js still low.

Standards by XCKD Standards by XKCD

Adoption

To measure adoption we can use some popular NPM packages that implement ESM only support as a proxy.

Looking at the package ansi-styles, with over 325 million weekly downloads (as of 29 January 2024), we can see that there are only approximately 22.5 million weekly downloads (as of 29 January 2024) of the ESM only version (v6) of the ansi-styles package. This is just under 7% of total downloads.

Looking at download numbers on a per version basis, the majority of downloads are for v4.30 of the package, which is not the last available CommonJS version. Even downloads for the last v3 patch version of the package are higher than the downloads of v5, the last available CommonJS version.

Comparing download numbers for the last CommonJS version (v5) at under 50 million weekly downloads versus the ESM version (v6) at approximatley 22.5 million weekly downloads it's an improved 45% of downloads. My assumption for this, is that for new projects developers are more likely to use ESM, but for older projects they aren't going to refactor to ESM.

This pattern continues with most packages that have migrated to ESM only. With adoption at under 10% of downloads for packages that have shipped ESM only versions we can see that the introduction of ESM into Node.js is still poor. Over three years since it's introduction ESM usage in Node.js appears to be primarily in new projects.

Why?

Why is adoption of ES Modules by the Node.js community important? What benefits will it have? The easy answer is that having one standard for all environments that run JavaScript is good. It will improve interoperability, developer productivity and performance.

The more tangible reason is that after more than three years of two module systems being available in Node.js, the ecosystem is worse off. This is to the detriment of developer productivity, interoperability and performance. Standardising on one module system will ease the pain, improving productivity and performance.

The desire to maintain backward compatibility with CommonJS, with the introduction of ES Modules, and to avoid a "Python moment" is holding Node.js back.

At a surface level you could say backwards compatibility with CommonJS was achieved. The long term unintended consequences of the decision to run two module systems has resulted in the same thing happening in slow motion. ESM has created a clear break in the community.

We see evidence of this split in the rise of dual module packages. Maintainers will ship both CommonJS and ESM code as they look to meet the needs of as many developers as possible. This has been doubling the size of packages shipped to NPM, and introduced sometimes complex build steps.

Dual module packages are not implemented consistently either. There is variable adoption of the exports field in package.json to define entrypoints, alongside the use of the unofficial module field in package.json to denote an ESM entrypoint. This leads to compatibilty problems and issues with bundlers.

There are also bizzare requests to put string formatting functionality into the platform core due to the ansi-styles library being ESM only.

On the flip side we could see a significant breaking change to users of the CommonJS version of node-fetch in future LTS versions of Node as punycode has been removed from the Node.js core which the CommonJS version of the library relies on.

Throw TypeScript into the mix, where ESM syntax is the popular choice for handling modules, but CommonJS is still the default transpilation target. We have the crazy situation of developers writing ESM like syntax, but shipping CommonJS, because it's default, not because it's preferred.

The pain caused by the dual module system has led to runtimes like Bun providing the ability to require() and import modules from the same module, diverging even further away from existing standards.

These are "edge cases" that impact millions of downstream apps and packages. The decision to maintain both module systems has increased complexity across the board.

Factor in that ESM is the module system for the web and all other non-server side JavaScript environments, alongside the rise of full stack development, it seems crazy to maintain a seperate module system for server side JavaScript. The goal should be a ubiquitous module system across all JavaScript environments.

Ophidiophobia

The overwhelming fear of replicating Python's big breaking change when upgrading to v3 is driving the desire to maintain both module systems. This ignores that the change only had long term benefits for the Python runtime and community. Python today is powering the current LLM wave, and has been the defacto language for ML and big data workloads for years. It is also the largest server side scripting language in the world today. Would it be so ubiquitous without the breaking changes that moved the language forward?

Ophidiophobia (or ophiophobia) is a specific phobia (irrational fear) of snakes - Wikipedia

Forcing breaking changes that move tech forward should not be seen as a bad thing. CommonJS evolved from the need for a module system for server side workloads. There was a greater need for a module system that encompassed all JavaScript runtime environments, and so the EcmaScript module spefication was created. Informed by the learnings from CommonJS, but expanded for all scenarios. It's time to say good bye to CommonJS as its holding the larger JavaScript ecosystem back.

Maintaining two module systems, and persistently having to debate and implement workarounds for the edge cases brings added load to the maintainers, and increased frustration for developers. Downstream it creates greater work for package maintainers trying to ship a version of their code for each module system. A single module system would will reduce package sizes, simplify CI/CD pipelines, remove build steps, improve performance and result in better developer productivity.

Lessons Learned

It appears the wrong lessons were learnt from the Python moment. There is a clear desire from the Node.js TSC to avoid it all costs. Rather than accepting you can't move a runtime forward without breaking changes, and some changes break more than others.

The lesson to learn was how to do it properly, not to avoid it at all costs. The Python community learnt a lot in getting the migration over the line. Key lessons learnt included

  • Automated tooling like the following packages
    • futurize or modernize to automatically convert Python2 code to Python3
    • caniusepython3 to detect which packages you use support Python3
    • Linters that could detect conformance with Python3
  • Clear messaging on end of life support
    • Setting a date when support would end for Python2, creating a clear incentive for laggards to migrate
    • Extended period to migrate. End of life support from Python2 was 12 years after the release of Python3
    • Defacto Python3 became the default install of Python.
  • Migration Guides
    • Python team created clear guides on how best to migrate.
    • The community stepped up and created multiple migration guides.
  • Wide package support
    • Early migration efforts were derailed by Python packages not having support for Python3
    • Improved package support unlocked migration for most users

This is not an exhaustive list. The Python community learnt how to handle it, achieved the migration and the runtime is stronger for it.

The Way Forward

Long term having one module system for all JavaScript environments is to the benefit of the larger JavaScript community. The best way to get there is to deprecate CommonJS. What is the path to deprecate CommonJS? Deprecation will be a forcing function. It will create a need for developers and businesses to migrate to ESM to remain on a supported runtime.

We know that the Node.js TSC is not against deprecating features that break packages. It gets done with almost every release. The most recent example is the deprecation of punycode from Node.js core. There needs to be an active decision to do the same with CommonJS, whilst acknowldeging the impact of the deprecation and mitigating it sufficiently.

To mitigate the impact, and achieve a responsible deprecation the following areas need to be addressed

  • Tooling
  • ESM by Default
  • Extended LTS for last CommonJS runtime
  • Documentation and Guides
  • Messaging

Tooling

Ensure there is adequate tooling to ease migration from CommonJS to ESM. Effectively create Node.js versions of futurize and caniusepython3. There are already linters that support ESM, so the tooling is already partially there. This will allow migration to be a more trivial exercise, reducing any potential risk.

ESM by default

Move Node.js to use ESM by default. There is already work under way to make this happen. The introduction of the --experimental-default-type flag in Node.js v21 is the first step towards this goal.

This is a great step, but there is still discussion as to when to move this from a feature flag to the default behaviour. Right now there is a compromised decision to skirt around the topic by shipping ESM detection on ambigous files. This which will slow things down for the average user. Unfortunately the TSC has chosen compromise over clarity for now, which will once again hurt the wider community.

Extended LTS for last CommonJS runtime

It took Python 12 years to remove support for Python2. The Node.js TSC should take a similar approach. Shipping a final version of Node.js with CommonJS support that has extended support beyond the normal support periods. This will give laggards the extended time needed to update their code bases.

Documentation and Guides

Create guides on migrations and how to handle edge cases. There are no official Node.js guides today on how to migrate a CommonJS code base to ESM. Many of the TSC discussions today on how to resolve the conflict between CommonJS and ESM could be resolved with clear and concise guides on the best ways to migrate or use mixed module systems in a staged migration.

There is also the structural issue that Node.js documentation only covers Node.js features, not the full scope of JavaScript. Understandably the Node.js team don't want to maintain documentation for all of JavaScript. The problem this creates is developers consistently hopping between Node.js documentation and JavaScript documentation (most likely MDN Web Docs).

Universal JavaScript documentation that covers Node.js and web runtimes would be hugely beneficial. This would reduce switching between docs, shortening the learning curve, and allow for more comprehensive documentation. In my opinion the best resolution to this is to incubate a global documentation project as part of the OpenJS foundation in collaboration with Mozilla and Node.js.

A clear official migration guide, not just release notes and general documentation, combined with universal JavaScript documentation will resolve a number of knowledge gaps. In addition once a decision has been communicated, you can expect a ton of community contributed blog posts that will reiterate the official guidance.

Messaging

Clear and concise messaging on the route to deprecation is needed, with clear timelines and milestones. Ambiguity in messaging hurts. The Node.js TSC is a technical team that guide the future of the Node.js, a separate team should be formed to handle the messaging around deprecation.

A potential timeline for deprecation should be measured in years, with six key milestones.

  • Creation of migration tooling, and updated documentation
  • ESM by default
  • Release of last LTS version with CommonJS support
  • Release of first LTS version with CommonJS deprecated via warning, but functionality still in core
  • Release of first version with CommonJS removed
  • End of LTS support for last CommonJS version

A reasonable timeline for this would be 6 to 8 years. If this is communicated in a clear understandable way it is very achieveable.

The Desire to Act

There are clear paths to deprecation, informed by lessons learnt from similar situations, what is missing is a desire to act. The ES Module project's focus is creating guides, documentation and tooling around ES Modules independently of the Node.js TSC or any other official OpenJS foundation structures. The rest lies in the hands of the Node.js TSC.

It will take a lengthy amount of time to deprecate CommonJS, and it all starts with a decision to do it. Hopefully this post sparks enough debate to get us closer to that decision.

About the authors