Slow rebuild whis many async chunks

[SuperOleg39]

Bug report

Hello!

Have a huge size application, with 180+ dynamic imports (a lot of pages).

Saw a very slow rebuild, after profiling found one of a build performance bottlelneck where method getAllAsyncChunks are called.

CPU-20221202T124255.cpuprofile.zip

Actual Behavior

On any change in any file in application, mini-css-extract-plugin spend a lot of time to create hashes

Expected Behavior

mini-css-extract-plugin create hashes only for changed files

How Do We Reproduce?

Unfortunately, project is in private repo.

We are using webpack@5

[alexander-akait]

Can you profile more and look what is slow on webpack side here? Also can you provide versions (latest?)?

[SuperOleg39]

Can you profile more and look what is slow on webpack side here? Also can you provide versions (latest?)?

Looks like the initial cause is how webpack visit modules for dynamic imports, this code in buildChunkGraph creates a very long queue.

For a server build, we use babel-plugin-dynamic-import-node, so every dynamic import() become static and this problem doesn't exists.

For client build, we can't do this, because need to have all of this components in stats, and our loadable components should works.

webpack@5.64.4

[SuperOleg39]

Interesting fact, sideEffects was disabled in our development build, and when we delete this option, default are true for this option, and rebuild time was improved from 10 to 1.5-2 seconds on my machine.

Need to make more tests, but looks like it will resolve our problem)

[alexander-akait]

Yeah, sideEffects is take a time, anyway the value is very big and should be less, can you try to update webpack to the latest version, maybe it is already fixed

[SuperOleg39]

Yeah, sideEffects is take a time

Interesting thing here is that sideEffects: true boost rebuild performance

can you try to update webpack to the latest version, maybe it is already fixed

Okay, thanks!

[alexander-akait]

Interesting thing here is that sideEffects: true boost rebuild performance

In this case webpack removes unused modules, so graph is less, so less time on it, maybe we have a sutiation where you can a lot of extra and unnecessary modules, sometimes it happens, especial in big projects

For fast solution - do you use cache (filesystem)? It will improve perf even in development mode (even with a lot of modules which will be dropped in future)

[SuperOleg39]

In this case webpack removes unused modules, so graph is less, so less time on it, maybe we have a sutiation where you can a lot of extra and unnecessary modules, sometimes it happens, especial in big projects

Have the same thoughts)

do you use cache (filesystem)?

Yes, must have feature.
And we are looking for lazyCompilation, but this feature still not works with SSR and loadable - webpack/webpack#15698

Thanks for the answers!

[alexander-akait]

@SuperOleg39 Wierd, it should work... Can you create small example? I will look, also you can disable it for SSR, but need to look at configuration (we can do it when target is not web on the plugin level, it will be very small plugin)

[SuperOleg39]

Can you create small example?

There is a ready for reproduction repo

I will look, also you can disable it for SSR, but need to look at configuration (we can do it when target is not web on the plugin level, it will be very small plugin)

We have a two parallel builds for browser and server, and if we will enable lazyCompilation for only node target, result speed will be the same for developers

[alexander-akait]

We have a two parallel builds for browser and server, and if we will enable lazyCompilation for only node target, result speed will be the same for developers

I mean only for the web target, anyway the node target should work too... I will look again, thank you

[SuperOleg39]

Thanks a lot!