This is a year-long summary of webpack5 upgrade practice. I started this project in April 2020 when webpack5 was in beta phase, and then I made a few more attempts, all of which were stuck in different branches due to compatibility between self-published and third-party plugins.

The following is a record of the practice related to the upgrade after it was finally completed in the project.

Upgrade of webpack and dependency versions

Install the latest version of webpack.

1
2
3
4

# 早期
yarn add webpack@next -D
# webpack5 正式发布后
yarn add webpack -D

Plugin Dependency Upgrade: Update the relevant plugins used in the project to their latest versions on demand. There may be various compatibility issues in this process, which need to be analyzed and solved according to the actual situation.

Upgrade dependency versions with npm-check-updates

Question: 160+ Third-party dependencies, how to quickly confirm the latest version and whether it needs to be upgraded?

Options.

  • npm outdated and npm update are too simple and brutal
  • Manual update: In Microsoft Visual Code editor, when you hover over a dependency line and hold it, it will look up the latest version of that dependency line.
  • Helper tool: npm-check-updates

Install and use npm-check-updates :

1
2
3
4
5

# 全局安装
npm i -g npm-check-updates
ncu
# 或者使用 npx
npx npm-check-updates

Example usage of the npm-check-updates command.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# 交互式升级：每一个依赖的修改都需手工确认
ncu -u -i
# 只更新补丁版本 -- 按指定的<target>级别更新。target 可选值：latest, newest, greatest, minor, patch
ncu -u --target patch
# 仅更新 devDependenties 下的依赖。dep 可选参数： prod, dev, peer, optional, bundle（多个可用逗号分隔）
ncu -u --dep dev
# 仅更新包名称包含 react 字符串的依赖
ncu -u --filter /react/
# 不更新包名称包含 electron 字符串的依赖
ncu -u --reject /electron/
# 检查更新是否可以通过测试(npm test)，并指定项目管理器为 yarn
ncu -u --doctor -p yarn

Project Upgrade Practice.

1
2
3
4
5

# devDependencies 依赖：全部快速升级至最新(types 类型定义等少数无需升级的手动修正还原)
ncu -u -p yarn --dep dev
 
# dependencies 依赖：交互式逐个确认
ncu -u -i -p yarn --reject /react/ --dep prod

Writing webpack configuration with TypeScript

It is recommended to use TypeScript to write webpack configuration files. With TypeScript’s type detection, you can quickly determine which property configurations are valid, no longer valid, or incorrect. Example.

1
2
3
4
5
6

import webpack from 'webpack';
const config: webpack.Configuration = {
  name: 'lzwme',
  mode: 'development',
};
webpack.watch(config, (err, stats) => {});

Start webpack.

1
2
3

ts-node -T webpack.config.ts
#或者
node --max_old_space_size=8192 -r ts-node/register/transpile-only webpack.config.ts

Of course, for js-formatted configuration files, you can also add @ts-check to the top of the configuration file to enable TS type checking capability for js files. Example.

1
2
3
4

// @ts-check
 
/** @type {import('webpack').Configuration} */
const config = {};

webpack build configuration updates and compatibility

  • Coding quality and efficiency
    • ESLint upgrade
    • husky upgrades: git hook performance is greatly improved in large projects
    • Public modules, ESLint / TS Check related
      • combokeys -> Shortcut coding TS refactoring
      • More…
  • webpack configuration and related dependency plugins
    • Custom plugin updates and development: BuilderManager, Benchmark, PluginsBuilder, caseSensitiveSassLoader, etc.
    • html-webpack-plugin : API changes, custom logic debugging analysis and compatibility updates that depend on its API
    • css-loader and cssModule changes
    • CopyWebpackPlugin usage changes, parameter configuration changes
    • webpack.IgnorePlugin parameter configuration changes
    • DLL-related configuration
    • More…
  • Remove the webpack plugin that is no longer used
    • script-ext-html-webpack-plugin : Support for feature requests in the latest version of the html-webpack-plugin plugin
    • thread-loader, cache-loader : change to use webpack5’s built-in caching and multithreading configuration, etc.
    • file-loader, url-loader
    • More…
  • Business dependency plugin compatibility
    • Huge impact, may lead to unpredictable accidents: minimal changes, confirmed by the business caller on an application-by-application basis

Compatibility details are the most complex and the problem is quite time-consuming to analyze, which is the main reason why it is difficult to upgrade smoothly. Heavy reliance on third-party libraries can deepen the complexity of project upgrades.

Compatibility of webpack plugins

Generally speaking three-party plugin compatibility is the main issue with upgrading webpack5. Early on, the degree of compatibility follow up varied across plugins, and there were quite a few details of odd performance issues.

  • webpack API changes, requiring development of plugin-compatible versions. Because some APIs were changed or deprecated, many plugins needed to be partially rewritten for compatibility.
  • Self-coded plugin compatibility. Basically all webpack plugins that are self-coded for specific needs need to be interface-level compatible, which increases the development and debugging effort significantly.
  • Large differences between old and new versions of three-way plugins. Some plugins may have large changes in the default parameter behavior in new versions, which can also lead to a variety of strange problems, debugging is more complex, and requires careful step-by-step analysis. For example, the problem of css-loader changing the default handling of cssModule.

css-loader 与 cssModule

The options.modules parameter needs to be configured to keep the logic consistent with previous versions of css-loader@2.

1
2
3
4
5
6
7
8

{
    loader: require.resolve('css-loader'),
    options: {
        importLoaders: 1,
        modules: { mode: 'global', },
        // localIdentName: '[name]__[local]__[hash:base64:5]',
    },
},

Changes to esModules default rules (standard ESM).

1
2
3
4
5

// 旧的用法失效
import * as styles from './style.scss';
 
// 批量替换为
import styles from './style.scss';

output err error problem

1
2
3
4
5
6
7
8

output: {
    path: pluginDesc,
    chunkFilename: '[id].chunk.js',
    filename: '[name].js',
    // @see https://github.com/webpack/webpack/issues/11660
    // chunkLoading: false,
    wasmLoading: false,
}

Deprecated file-loader and url-loader

DEPRECATED for v5: please consider migrating to asset modules.

  • raw-loader: to import a file as a string
  • file-loader: handle file dependencies loaded by import/require etc.
  • url-loader: relies on file-loader, contains all its capabilities, and can convert files to base64

Resource processing in webpack5 asset modules:

  • asset/source: exports the source code of the asset. Previously achievable by using raw-loader.
  • asset/resource: emits a separate file and exports the URL. Previously achievable by using file-loader.
  • asset/inline: exports a data URI of the asset. Previously achievable by using url-loader.
  • asset: automatically chooses between exporting a data URI and emitting a separate file. Previously achievable by using url-loader with asset size limit.

Using url-loader in webpack5

If the url-loader dependency of an old project is difficult to modify and remove, there is still a corresponding configuration that can be implemented

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

module.exports = {
  module: {
    rules: [
      {
        test: /\.(png|jpg|gif|svg)$/i,
        // 避免被 asset modules 处理
        dependency: { not: ['url'] },
        // 避免 webpack5 重复处理
        type: 'javascript/auto',
        use: [
          {
            loader: 'url-loader',
            options: { limit: 8192 },
          },
        ],
      },
    ],
  }
}
Re

asset modules configuration and usage examples

If possible, using asset modules directly is the best way to be future-proof.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

module.exports = {
    entry: './src/index.js',
    output: {
        filename: 'main.js',
        path: path.resolve(__dirname, 'dist'),
        // 自定义输出路径 -- urlLoader.outputPath
        assetModuleFilename: 'images/[hash][ext][query]',
    },
    module: {
        rules: [{
            test: /\.(png|jpg|gif|svg)$/i,
            type: 'asset/resource',
        }, {
            test: /\.html/,
            type: 'asset/resource',
            // 配置独立的输出路径
            generator: {
                filename: 'static/[hash][ext][query]',
            },
        }, { // 文件内联
            test: /\.svg/,
            type: 'asset/inline',
        }, { // 根据文件大小采用不同处理 -- url-loader
          test: /\.txt/,
          type: 'asset',
          parser: {
              dataUrlCondition: {
                  maxSize: 4 * 1024 // 4kb
              }
          }
      }],
    },
};

About sass: node-sass and dart-sass

  • sass : css extension language
  • node-sass : C++ module with system platform and Node.js version dependency
    • Pros: high performance, fast compilation speed
    • Disadvantages: installation failure due to high platform dependency
  • dart-sass : dart written and compiled as wasm module
    • Pros: cross-platform, less problems with installation, etc., has become the main official choice
  • sass-loader has changed the default interpreter to dart-sass

Can I switch directly to dart-sass now?

Business code compatibility updates are cumbersome and require extensive regression testing.

Building Performance Efficiency: Persistent Caching and Multithreading

The most time consuming part of the webpack build process is the compilation of the various source files. The main ideas to improve build efficiency are multi-threaded parallelism and avoid duplicate compilation (caching of compilation results).

Build performance optimization for webpack4 and earlier

Long-term development and maintenance of large projects often take tens to hundreds of seconds to build. Before webpack4, plugins like happypack, hard-source-webpack-plugin, thread-loader, cache-loader, etc. were almost standard for project builds in order to use caching to improve build efficiency, but even then it was often unsatisfactory, as these third-party plugins could not work together perfectly and integrate into all parts of the build process.

  • cache-loader: cache build results to node_modules/.cache-loader directory (you can specify the location)
  • Enable the babel-loader configuration item: cacheDirectory: true, similar to cache-loader
  • hard-source-webpack-plugin: cache build results to the node_modules/.cache/hard-source directory (you can specify the location)
  • thread-loader: execute the loader build process using multiple threads
  • happypack : An early multi-threaded solution implemented by a third party, not updated since the official thread-loader was released.
  • webpack dll : Pre-compiled resources that are not frequently modified
  • more…

cache configuration in webpack5

In webpack5, the cache configuration is implemented through the cache field, and it performs better.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

{
  // The number of parallel processed modules in the compilation.
  parallelism: require('os').cups().length -1,
  cache: isProd
      ? { type: 'memory' }
      : {
            type: 'filesystem',
            name: `test-${subModuleName}`,
            version: package.version,
        },
}

Comparison of webpack5 caching effects

webpack5 first build.

webpack5 secondary build (with caching).

Comparison of webpack4 caching effects

webpack4 first build (with thread-loader).

webpack4 secondary build (with caching).

Go further: thread-loader and esbuild-loader

In webpack5, the thread-loader still has its place. While parallelism simply specifies the number of threads for parallel builds, thread-loader allows for separate multi-threaded build testing and optimization for special loader classes such as ts-loader.

In addition, using esbuild-loader instead of babel-loader to achieve compilation acceleration is also an option to speed up build efficiency. The pros and cons lie in the dependency of the build output and development experience on their respective ecologies, where the difference in hot update is a big contrast in the development experience.

webpack5 first build (thread-loader).

Efficiency of next generation build tool solutions

The development experience is undoubtedly quite comfortable for next-generation front-end build solutions like vite and snowpack with no Bundles. Thanks to the no-build, compile-on-demand feature, projects can be launched in seconds, no matter how large they are.

However, from simple tests and community feedback, there are many details that are difficult to handle perfectly when dealing with complex and personalized requirements. webpack has a long way to go.

A brief overview of other configuration updates for webpack5

optimization

Optimization is an important configuration property that has changed significantly in webpack5. Some optimization features that used to be enabled using the built-in plugins have been turned on or off as configuration items here. See here for details.

https://github.com/webpack/changelog-v5/blob/master/MIGRATION%20GUIDE.md#update-outdated-options

node environment compatibility

Previously, the goal of webpack was to build applications that could run browser-side, so there was default compatibility support for some variables that were only available in the Node.js environment. In webpack5, these compatibility handles are no longer supported by default. Now the node property can only be configured with a single global property, and all other compatibility is achieved through resolve.fallback itself. For example, for compatibility with nodejs native modules such as fs, path, buffer, etc.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

{
  node: {
    global: true,
  }
  resolve: {
    fallback: {
      crypto: false,
      fs: false,
      child_process: false,
      electron: false,
      // 不作处理：例如对运行环境有检测逻辑，只在 node/Electron 环境下才会实际执行调用
      path: false,
      // 浏览器端兼容
      // path: require.resolve('path-browserify'),
      buffer: require.resolve('buffer/'),
    }
  }
}

A value of false means that nothing is done, which is basically similar to the effect of dll filtering in externals.

Compatibility with Buffer also requires the ProvidePlugin plug-in.

1
2
3

new webpack.ProvidePlugin({
    Buffer: ["buffer", "Buffer"]
})

Warning message for deprecating APIs

For some deprecating APIs, webpack5 wraps them with util.deprecate. It makes these APIs print a warning message on the command line when they are called.

During the upgrade to webpack5, you should set process.traceDeprecation=true or add the command line argument -trace-deprecation to debug to see where API calls are deprecated and try to update or use the latest version of the plugin in question.

If the latest versions of some plugins you must use also do not handle such API calls yet, you can set -process.noDeprecation=true to turn it off: -process.noDeprecation=true.

Other

  • webpack5 defaults some common configurations, such as entry: 'src/index.js'
  • If you are using Yarn PnP and have pnp-resolver-plugin configured, then you can remove the relevant configuration, which is now built-in.

You can start by referring to the official webpack4 -> webpack5 migration guide at

https://github.com/webpack/changelog-v5/blob/master/MIGRATION%20GUIDE.md

Next generation build tool?

The main popular build tools currently available are.

  • webpack webpack is a static module bundler for modern JavaScript applications
  • rollup Rollup is a module bundler for JavaScript
  • Parcel Extreme zero-configuration web application packaging tool
  • Snowpack The faster frontend build tool.
  • vite The next generation frontend development and build tool

Javascript / typescript compilers.

  • babel Babel is a JavaScript compiler
  • tsc(typescript)
  • esbuild Written in Go and mainly used for build acceleration
  • swc Super fast javascript / typescript compiler. written with Rust

The core idea of Snowpack and vite, currently known as the next generation of build tools, is based on the ES20200 specification’s ES Modules native support, where the build process essentially compiles the source code into the ES Module specification format and directly references it in the browser.

1

<script type="module" src="index.js" ></script>

Key features/objectives.

  • No bundles, true compile-on-demand, load-on-demand
  • Extremely fast cold start
  • Extremely fast hot updates
  • Significantly improved development efficiency and experience
  • More…

Main problems.

  • Compatibility, compile result differences: migration of old projects is risky
  • Some feature usage/syntax not easy to convert support
  • Not enough richness of plug-in support
  • Cost of self-coded plug-in development
  • More…