RFC: Using Performance panel for Node.js / Deno JavaScript CPU Profiling #2

jecfish · 2022-11-15T06:32:49Z

jecfish
Nov 15, 2022

Author: Nancy Li (@lilysjtu2011), Jecelyn Yeen (@jecfish)
Posted: Nov 9, 2022
Status: Complete
Chrome bug tracker: https://crbug.com/1354548

The goal of this RFC is to validate the design with the community, solicit feedback on open questions, and enable experimentation via a non-production-ready prototype included in this proposal.

Complete: This RFC is now complete. See a summary #2 (comment).

Motivation #

Since Chrome 58, the DevTools team set a long-term goal to deprecate the JavaScript Profiler (Profiler tab), and migrate Node.js and Deno developers to use Performance panel for profiling JavaScript CPU performance.

These changes allow users to profile performance consistently with one single entry point - the Performance panel.

Goals #

Profile Node.js / Deno JavaScript CPU performance with Performance panel
- Developers are able to profile JavaScript CPU performance with the Performance panel.
- The Performance panel should display relevant JavaScript CPU information, similar to the Profiler tab.
Minimize impact on the ecosystem
- Overall mental model: developers should be able to apply their existing knowledge to profile JavaScript CPU performance using the Performance panel, with minimal learning curve.

Current state #

Here is the current workflow on profiling JavaScript CPU performance:

Run node --inspect file-name.js or deno --inspect file-name.js (e.g. use this example file)
Open DevTools for Node via one of these options:
- The green Node button at the top of DevTools
- Enter url chrome://inspect > Devices > click Open dedicated DevTools for Node
- Enter url chrome://inspect > Remote target > click Inspect
Start and end profiling by clicking the Record button in the Profiler tab.
View the result in the Profiler tab. Use the dropdown to switch to different views. It support 3 modes currently :
- Chart. Users can switch to the Chart view with the dropdown.
- Heavy (bottom up). This is the default view.
- Tree (top down). Users can switch to the Chart view with the dropdown.
Chart

Heavy (bottom up)

Tree (top down)

Try the prototype: Profile using the Performance panel #

There is a prototype implementation. Below, we show a guide for trying it out. This is a draft implementation, with missing features and non-final design aspects.

Please use Chrome Canary to play with the prototype.

Profiling with the Record button

Run node --inspect file-name.js or deno --inspect file-name.js
Open DevTools for Node. (Chrome Canary)
Open the Performance panel via the 3-dot menu > More tools > Performance panel. This step is not required once the changes are rolled out officially, the Performance panel will show by default.
Start and end profiling by clicking the Record button in the Performance panel.
View the result in the Performance panel. The UI is different from the Profiler tab but you can find all the information:
- Chart. The Performance panel always shows the chart on top.
- Heavy (bottom up). The Performance panel shows this information in the bottom pane > Bottom-up.
- Tree (top down). The Performance panel shows this information in the bottom pane > Call tree.
Chart

Bottom-up

Call tree

Profiling with the console.profile() command

If you profile performance with the console.profile() command, you need to enable an experiment to map the result to the Performance panel.

Enable the View console.profile() results in the Performance panel for Node.js checkbox via the Settings icon > Experiments tab.
Open DevTools for Node. (Chrome Canary)
Run node --inspect file-name.js or deno --inspect file-name.js (e.g. use this example file)
Once the profile is completed, the result will be shown in the Performance panel automatically.

FAQ #

What will happen with the Profiler tab in the future? Will that be deprecated / removed?
Yes, we plan to deprecate the Profiler tab by stages. We will look into community feedback, and implement the JavaScript profiling capability into the Performance panel before deprecation.
Can I load the CPU profile I saved in the Profiler tab previously into the Performance panel?
Yes. You can import .cpuprofile files to the Performance panel.
Are both Profiler tab and Performance panel using the same data source?
Yes. The backend data source is the same for both. However, there are frontend logics that change the displayed data in the Profiler tab.

Below are the same profiling results shown in the Profiler tab and Performance panel.

There are 3 1 known differences at the moment:

a. From showing frames directly to organizing them under call stack - In the above example, the Profiler shows executeUserEntryPoint and the Module frames straight away. On the other hand, the Performance panel organizes them under the same call stack, you need to expand them to view it.
b. ~~crbug.com/1382733 (Fixed) - Some frames like (garbage collector) are hidden now in the Performance panel. We are working on adding them.~~
c. ~~crbug.com/1080918 (Fixed) - Differences in profiling setTimeout call due to the stack trace collection. We are working on addressing this.~~
Where is the realtime total JS heap size view?
The realtime Total JS heap size information is already available in the Memory panel. There is no plan to move that into the Performance panel.

The realtime total JS heap size in the Memory panel
Does the Performance panel support reload and record Node.js / Deno performance?
No. This proposal focuses on migrating the existing features in the Profiler tab to the Performance panel. The Profiler tab doesn’t support reload and record.

Questions for Discussion #

In addition to general feedback, we would like to collect feedback on the following specific questions:

Does the prototype correctly display JavaScript profiling information?
We are interested in any cases where the information is missing or has discrepancy with the Profiler tab.
Does profiling in the Performance panel UI work well for you?
We would be interested in any use cases where this change in UI would be problematic or burdensome, and where the UI is important.
For example, should we enlarge the Bottom-up tab and show it by default when you open the Performance panel in Node.js / Deno mode?

hashseed · 2022-11-16T08:37:12Z

hashseed
Nov 16, 2022
Maintainer

@joyeecheung @mhdawson @mcollina @jasnell @gireeshpunathil @Qard FYI

3 replies

Qard Nov 17, 2022

Looks very nice!

You mention there's no plan to move Total JS heap size, but the performance tab includes memory information currently. Is there some technical reason for not moving Total JS heap size too or just a scope concern and could therefore happen separately in the future? I would really like to see a more unified view for that. Something more like macOS Instruments would be really nice where you could pick which profilers to activate and it will capture and visualize them all together.

lilysjtu2011 Nov 21, 2022

Hi @Qard , the total JS heap size in the RFC referring to the real time total JS heaps size view (as highlighted below).

We don’t plan to migrate that to the Performance panel because the Memory panel has a similar UI for that.

Are you referring to another “Total JS heap size” or you would like to see this information in the Performance panel?

Qard Nov 21, 2022

Yes, that's what I was referring to. The Performance tab has a checkbox for memory and I'm unclear what, if anything, is stopping the existing memory tab data from being ported into that.

jecfish · 2022-11-16T21:29:10Z

jecfish
Nov 16, 2022
Author

@jrieken @bartlomieju fyi

0 replies

gireeshpunathil · 2022-11-17T03:48:34Z

gireeshpunathil
Nov 17, 2022

@RafaelGSS fyi

0 replies

joyeecheung · 2022-11-17T22:18:35Z

joyeecheung
Nov 17, 2022

Nice write-up of the new performance panel! I am wondering, is the Node.js-dedicated panel going to remain in chrome://inspect?

Also, I gave it a try and profiled a typical HTTP server:

'use strict';

const http = require('http');
const bodyLength = 12345;

const body = 'c'.repeat(bodyLength);
const server = http.createServer(function(req, res) {
  res.writeHead(200, {
    'Content-Length': bodyLength,
    'Content-Type': 'text/plain'
  });
  res.end(body);
});
server.listen(0, '127.0.0.1', () => {
  const { address, port } = server.address();
  console.log(`http://${address}:${port}`);
});

And profiled it with:

$ autocannon -d 10 http://127.0.0.1:$PORT

Here is the comparison (the workload in the particular span that the profile is taken differed slightly so minor differences in the calculations can be ignored):

The call trees:

The bottom-ups:

Close-up looks of some frames:

Other than the (garbage collector) entries missing, as already mentioned in the OP, there are some noticeable differences:

(idle) is missing
(program) is missing - like (idle) or (garbage collector) this is actually in the inspector protocol response.
There is now a Function call entry, is there some documentation about what that is? It looks like some sort of "native-calling-back-to-JS trampoline" entry.
I can see in the summary that there are System and Idle entries in the chart, but they are quite different from the calculations in the profiler panel - AFAIK the calculations from the profiler panel are correct (if we replace System with program, which in V8 means a combination of EXTERNAL (embedder) + PARSER + COMPILER and a few other VM states in the "system").
It is now possible to see the call trees and bottom-ups of a specific span, which is pretty nice 👍
When I saved the profile from the performance panel, the resulted file is named Profile-<timestamp>.json, from the profiler panel it's CPU-<timestamp>.cpuprofile. The JSON saved from the performance panel is in a format different from the JSON-formatted response from the inspector protocol (that is, https://chromedevtools.github.io/devtools-protocol/tot/Profiler/#type-Profile). There are some external tools that analyzes the profile using the inspector protocol format, so to fully replace the original panel it would be nice to be able to save the profile in that format, maybe after answering a prompt before saving.

3 replies

lilysjtu2011 Nov 21, 2022

Hi Joyee, for your first question not sure if I understand your question correctly. Do you mean if we will remove the link in chrome://inspect? (See the image below)

The “Open dedicated DevTools for Node” link will remain showing. However, instead of showing the Profiler tab when clicked, it will show the Performance panel instead.

The Profiler tab will be removed in the future.

lilysjtu2011 Nov 21, 2022

For your detailed feedback, here are my replies:

Q1 & Q2: Missing (idle)(program) entries

These missing entries are caused by the same reason as (garbage collector). We are currently working on it. Here is the tracking bug crbug.com/1382733.

We will update again when the bug is fixed!

Q3: The Function call entry

After some investigation, this is actually related to Q1 & Q2 crbug.com/1382733. After the bug fix, the Function call should be gone.

Q4: Incorrect calculation in `System` and `Idle` calculations in the Performance panel’s Summary tab

We’ve opened a bug crbug.com/1392436 for this. I will investigate this further.

Q5: Ability to view Call tree and Bottom-ups on a specific span

Good to know. Thank you.

Q6: Options to download .cpuprofile from the Performance panel

Nice feature request. Created a new feature request crbug.com/1392446 to track this.

lilysjtu2011 Jan 2, 2023

Hey @joyeecheung , some updates here.

The Performance panel is updated so the system nodes like (idle), (garbage collector) and (program) are no longer missing. Also the time calculation is also corrected now.
And for downloading, you can now download .cpuprofile when you use the Performance panel for Node.js.

Please take a look at the updates and feedback is welcomed :)
Thanks

mcollina · 2022-11-18T11:11:33Z

mcollina
Nov 18, 2022

A use case I have been using is to collect CPU and memory profiles using the inspector API directly and then loading them up in chrome to visualize them. Given the format has changed, will this still be supported?

2 replies

jrieken Nov 18, 2022

Same here. In VS Code we automatically perform CPU profiling and analyse its data. For that we use Profiler.start and Profiler.end command (from https://chromedevtools.github.io/devtools-protocol/tot/Profiler/). So, is this a UX only change or will the Profiler also disappear from the devtools-protocol?

hashseed Nov 18, 2022
Maintainer

CDP is not going to be affected. We are talking about replacing one consumer of the protocol with another consumer, for the Node.js use case.

Qard · 2022-11-18T20:06:01Z

Qard
Nov 18, 2022

Seems to not handle large profiles as well as the old Profiler tab. I opened a ~25mb profile in the Profiler tab which loaded and displayed fine and could be navigated relatively smoothly with only a little bit of lag. I loaded the same profile in the Performance tab and it seemed to lock up while loading, eventually displaying the thing after several minutes. When I tried to click anything in the chart visualization it crashed the whole browser. 😬

5 replies

jecfish Nov 21, 2022
Author

Thanks Stephen, created a crbug.com/1391853 for @lilysjtu2011 to follow up.

Qard Nov 21, 2022

Thanks. I added a comment that I can share the profile data with V8 folks if they want to investigate that profile specifically. 😄

hashseed Nov 21, 2022
Maintainer

It's not actually V8 folks, but DevTools folks working on this.

So if you can share your profile with e.g. [email protected], we might use the Performance panel on the Performance panel so that we can profile the performance of the Performance panel loading your performance profile.

😄

jecfish Nov 21, 2022
Author

Hah, before this message, I've dropped @Qard a follow up email with @lilysjtu2011. :)

RFC: Using Performance panel for Node.js / Deno JavaScript CPU Profiling #2

Uh oh!

Uh oh!

Motivation #

Goals #

Current state #

Try the prototype: Profile using the Performance panel #

Profiling with the Record button

Profiling with the console.profile() command

FAQ #

Questions for Discussion #

Replies: 18 comments · 37 replies

Uh oh!

hashseed Nov 16, 2022 Maintainer

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jecfish Nov 16, 2022 Author

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hashseed Nov 18, 2022 Maintainer

Uh oh!

Uh oh!

jecfish Nov 21, 2022 Author

Uh oh!

Uh oh!

hashseed Nov 21, 2022 Maintainer

Uh oh!

jecfish Nov 21, 2022 Author

Uh oh!

Replies: 18 comments 37 replies

hashseed
Nov 16, 2022
Maintainer

jecfish
Nov 16, 2022
Author

hashseed Nov 18, 2022
Maintainer

jecfish Nov 21, 2022
Author

hashseed Nov 21, 2022
Maintainer

jecfish Nov 21, 2022
Author