Skip to content

Conversation

@maiieul
Copy link
Contributor

@maiieul maiieul commented Aug 31, 2025

This PR contains:

  • bugfix
  • feature
  • refactor
  • documentation
  • other

Are tests included?

  • yes (bugfixes and features will not be merged without tests)
  • no

Breaking Changes?

  • yes (breaking changes will not be merged unless absolutely necessary)
  • no

List any relevant issue numbers:

Description

Qwik needs to use manualChunks to achieve its automatic fine grained bundling and preloading of bundles. This PR would solve a major issue for us, which I explain in more details in #6086.

Sadly I think the PR in its current state would be a breaking change, but I'm not 100% sure. If it is, we (the Qwik team) believe the need for solving the underlying issue is big enough that it justifies creating a new API. I can update the PR (or create a new one). We haven't found any other good alternative to this problem. We were using the optimizer (our own rust compiler) to do so in the past, but that broke the Vite ecosystem (issues with Storybook for example) and was definitely not as good of a solution as what we can achieve with Rollup and this version of manualChunks.

@vercel
Copy link

vercel bot commented Aug 31, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
rollup Ready Ready Preview Comment Sep 19, 2025 1:47pm

@maiieul maiieul changed the title fix: unhandled manualChunks merged with the first manualChunk encountered alphabetically fix: manualChunks and non manualChunks shared dependencies are merged with the first manualChunk encountered alphabetically Sep 1, 2025
Copy link
Member

@lukastaegert lukastaegert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @maiieul, thank you for bringing this to our attention and suggesting a fix. I think you are right in so far as this would be possibly a breaking change for some users. I agree, though, that for nearly all situations where a user supplies a function for manualChunks, your approach is more correct.
The reason it works like this is that originally, we only had the object form API where you basically designate some "entry points" for your manual chunks. For most use cases, your change would make the object form of manualChunks unusable.
In the "function" form, however, your behavior is much closer to expectations I think, i.e. you manually take a few modules out of chunk assignment but let Rollup handle the rest.

Here would be a suggestion how to move forward:

  • We make the behavior I outlined the way the next major version of Rollup will work, i.e. when an object is used, we merge dependencies into the chunks, otherwise we do not. One way to do that is to start here

    rollup/src/Bundle.ts

    Lines 170 to 174 in 298609e

    const manualChunkAliasByEntry =
    typeof manualChunks === 'object'
    ? await this.addManualChunks(manualChunks)
    : this.assignManualChunks(manualChunks);
    const snippets = getGenerateCodeSnippets(this.outputOptions);
    where we specifically know which form was used. We could for instance store this in a flag that we pass to getChunkAssignments to control which version is used.
  • To allow a gradual transition, we add a temporary new option e.g. onlyExplicitManualChunksAssignments that we remove again in the next major version.
  • We also need to adapt the documentation for manualChunks and add some basic documentation for the new option.

Let me know if you would be willing to perform these changes and if you need any additional help.

Copy link
Contributor Author

@maiieul maiieul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @lukastaegert, thanks for the prompt reply!

I must say I had not thoroughly checked the snapshot results and the object form was broken indeed 😄.

Let me know if you would be willing to perform these changes and if you need any additional help.

Happy to give it a go! In fact, I already started diving into it 😁

We make the behavior I outlined the way the next major version of Rollup will work, i.e. when an object is used, we merge dependencies into the chunks, otherwise we do not.

I dug a little deeper into the object vs function form and found out that there are quite a few discrepencies between the two. To me as a consumer of Rollup, I would find it more intuitive if both behaved similarly. If we go down the path of merging dependencies into the chunks only when the object form is used, the discrepencies become even bigger, because now we have a "merge deps" behavior with the object form, and a "let Rollup handle the rest" with the function form.

I explored a way to make both APIs consistent, by converting the object form to a function and always use the new function form code path of assignManualChunks for both.

The idea is that when the developer passes

    manualChunks: {
      vendor: ['lodash-es', 'clsx'],
    },

Rollup then converts it to

    manualChunks: (id) => {
      if (id.includes('lodash-es')) return 'vendor';
      if (id.includes('clsx')) return 'vendor';
    },

and then always uses the assignManualChunks logic, without the addStaticDependenciesToManualChunk merge logic.

Doing this, both forms work exactly the same way, which means:

  • No surprises to the developer when switching between one form to the other.
  • The code to get the manualChunkAliasByEntry and later on getChunkDefinitionsFromManualChunks becomes quite a bit simpler.
  • The object form no longer automatically merges dynamic deps or handle "non-inlcuded" manualChunks and other edge-cases. It is up to the developer to precisely tell Rollup how to group the manualChunks together.
  • The two forms have the same algorithmic efficiency. I don't know if the object form was supposed to be faster than the function form. I have a feeling that removing the recursion in addStaticDependenciesToManualChunk should be faster but I might be wrong. I didn't run any benchmark on this. If the object form was faster, would you see the normalization of both forms as a problem? In my experience with Qwik apps, manualChunks in the function form is fast and never the bottleneck, even with a lot of conditional logic.

I went ahead and implemented the logic for you to see. I added a comment to each test (either on _config.js or _expected/es/...), explaining the new behavior and why the new output makes sense to me.

(I sent you a message on Discord in case you would like to go over it in a call)

Looking forward to hear from you. Once we are aligned on the code changes I will proceed with implementing the new onlyExplicitManualChunksAssignments option to be removed in the v5 release and updating the docs.

Copy link
Member

@lukastaegert lukastaegert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your idea and I actually like it. In this spirit, the array form could even be extended to support both strings and regular expressions, making it truly powerful.

With regard to the test changes:
Many tests that use manual chunks specifically rely on certain modules to be merged. So by default, I would rather change the manualChunk config in those tests so that the same merges take place than to change the expected generated output.

As for the way forward, progress is unfortunately slow with Rollup and I cannot promise a timely new major release. If you want the feature quickly, how about we add a flag e.g. "onlyExplicitManualChunks" or something that switches to the new algorithm until we do the full switch in the next major version? I would be fine with updating the tests already to specifically set the flag and just keep one legacy test for the old logic.

@codecov
Copy link

codecov bot commented Sep 13, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.74%. Comparing base (7471074) to head (9218c85).
⚠️ Report is 8 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6087   +/-   ##
=======================================
  Coverage   98.74%   98.74%           
=======================================
  Files         271      271           
  Lines       10641    10644    +3     
  Branches     2848     2850    +2     
=======================================
+ Hits        10507    10510    +3     
  Misses         89       89           
  Partials       45       45           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@maiieul
Copy link
Contributor Author

maiieul commented Sep 13, 2025

I agree with your idea and I actually like it. In this spirit, the array form could even be extended to support both strings and regular expressions, making it truly powerful.

Great! I'll proceed with the changes. I didn't think about the regular expressions. Good idea.

Many tests that use manual chunks specifically rely on certain modules to be merged. So by default, I would rather change the manualChunk config in those tests so that the same merges take place than to change the expected generated output.

Sounds good.

As for the way forward, progress is unfortunately slow with Rollup and I cannot promise a timely new major release. If you want the feature quickly, how about we add a flag e.g. "onlyExplicitManualChunks" or something that switches to the new algorithm until we do the full switch in the next major version?

Yes, that would be great. We can switch that on by default for Qwik consumers so it's fine to have it as a flag until the next major.

@maiieul
Copy link
Contributor Author

maiieul commented Sep 16, 2025

@lukastaegert I tried to make another PR for my idea of unifying the object and function forms implementation, but I closed it as it can lead to non-deterministic behavior: #6106 (comment).

As such, I think we should proceed with your orignal idea: "We make the behavior I outlined the way the next major version of Rollup will work, i.e. when an object is used, we merge dependencies into the chunks, otherwise we do not.". This does mean the behavior for the object form and function form will be even more different, but that we can explain through documentation. Another option that could be considered is to deprecate the object form entirely to avoid the inconsistency, because there's nothing theoretically that can be done with the object form that cannot be done with the function form, but I guess the feature is too widely used to be deprecated.

… with the first manualChunk encountered alphabetically
@maiieul maiieul force-pushed the fix-manualChunks-merging branch from ec902c8 to 4139221 Compare September 18, 2025 15:44
@maiieul maiieul marked this pull request as ready for review September 18, 2025 15:56
@maiieul
Copy link
Contributor Author

maiieul commented Sep 18, 2025

@lukastaegert this can be reviewed again. I went with the onlyExplicitManualChunks approach.

I only added one test to verify the new behavior. Do you think this will be sufficient?

Copy link
Member

@lukastaegert lukastaegert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice, thanks a lot. Only some minor comments, then we can release this!

@lukastaegert lukastaegert added this pull request to the merge queue Sep 19, 2025
Merged via the queue into rollup:master with commit 3f124ba Sep 19, 2025
44 checks passed
@github-actions
Copy link

This PR has been released as part of [email protected]. You can test it via npm install rollup.

dummdidumm added a commit to sveltejs/kit that referenced this pull request Oct 7, 2025
…sues, but they snuck back in.

The problem, it turns out (read https://rollupjs.org/configuration-options/#output-manualchunks very carefully), is that dependencies of a manual chunk might get pulled into that manual chunk, too. If the dependency happens to be our env module, this can cause issues since the module might eagerly access the env.

Luckily Rollup has recently solved this via a dedicated option (that will be the default in version 5 and is hence deprecated right away) named `onlyExplicitManualChunks` (rollup/rollup#6087). This solves our problems around chunk ordering. For users not on the latest version yet we do a best-effort fallback by extracting env into its own chunk. This should solve most issues people encounter but the general problem can still occur, which is only fixed with the new option (hence we warn if the rollup version is not up to date).

Fixes #14590
dummdidumm added a commit to sveltejs/kit that referenced this pull request Oct 7, 2025
…ules

With #14571 we hoped to have solved the chunk/code execution order issues, but they snuck back in.

The problem, it turns out (read https://rollupjs.org/configuration-options/#output-manualchunks very carefully), is that dependencies of a manual chunk might get pulled into that manual chunk, too. If the dependency happens to be our env module, this can cause issues since the module might eagerly access the env.

Luckily Rollup has recently solved this via a dedicated option (that will be the default in version 5 and is hence deprecated right away) named `onlyExplicitManualChunks` (rollup/rollup#6087). This solves our problems around chunk ordering. For users not on the latest version yet we do a best-effort fallback by extracting env into its own chunk. This should solve most issues people encounter but the general problem can still occur, which is only fixed with the new option (hence we warn if the rollup version is not up to date).

Fixes #14590
Rich-Harris pushed a commit to sveltejs/kit that referenced this pull request Oct 7, 2025
…ules (#14632)

With #14571 we hoped to have solved the chunk/code execution order issues, but they snuck back in.

The problem, it turns out (read https://rollupjs.org/configuration-options/#output-manualchunks very carefully), is that dependencies of a manual chunk might get pulled into that manual chunk, too. If the dependency happens to be our env module, this can cause issues since the module might eagerly access the env.

Luckily Rollup has recently solved this via a dedicated option (that will be the default in version 5 and is hence deprecated right away) named `onlyExplicitManualChunks` (rollup/rollup#6087). This solves our problems around chunk ordering. For users not on the latest version yet we do a best-effort fallback by extracting env into its own chunk. This should solve most issues people encounter but the general problem can still occur, which is only fixed with the new option (hence we warn if the rollup version is not up to date).

Fixes #14590
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants