(In reply to Eric Rahm [:erahm] from bug 1537694 comment #11)

(In reply to :Gijs (he/him) from 1537694 comment #5)

Separately, I wonder if it'd be productive to make the AWSY code never load the new tab page, and not preload it (from the fact that it opens a blank page on startup, and removes the preloaded browser after closing tabs, I assume that was originally the goal?) as long as we also have a separate, specific AWSY test for how much memory that new tab page takes up, so we don't lose sight of its impact.That would also make regressions/improvements in the new tab page itself show up properly, where they might not currently meet thresholds for reporting on the "main" AWSY tests. Eric, does that sound sane? How hard would it be to get to that stage?

That sounds reasonable. Someone would have to add a separate test that loads the new tab page. That should be pretty simple, it's basically just the startup portion of the test. Then probably a front-end/marionette dev would need to modify the test_memory_usage test to somehow open a page in a new tab without first opening a new tab to a blank page within the constraints of the marionette framework.

I'm not sure I'm the best person to own doing this work, so if someone else wants to take it I'm all for it. I'm guessing the simplest way to make this work is:

0. disable new tab page preloading, force the privileged content process on;
1. create a test that loads the about:home/about:newtab page on startup "naturally", ie the same way a "normal" Firefox profile would by default
2. measure the privilege content process memory then
3. add 5 more tabs that also load about:newtab, measure that (after full pageload etc.)
4. close the tabs again, GC, measure again.

Mike/Eric, does that seem like it'd work? Mike, any tips from bug 1529965 on how to ensure the new tab page has fully loaded? Eric, should we measure the main process and privileged content process separately (should we even measure both?)

Kate/Tim: heads up that this would measure about:newtab / AS memory use and so keep track of performance issues with new code in that area.

I'm going to redirect to Kate, who probably has a better sense of how Firefox knows when about:home has finished loading. (The test in bug 1529965 just polls Telemetry until the probe we care about shows up).

(In reply to :Gijs (he/him) from comment #0)

(In reply to Eric Rahm [:erahm] from bug 1537694 comment #11)

(In reply to :Gijs (he/him) from 1537694 comment #5)

Separately, I wonder if it'd be productive to make the AWSY code never load the new tab page, and not preload it (from the fact that it opens a blank page on startup, and removes the preloaded browser after closing tabs, I assume that was originally the goal?) as long as we also have a separate, specific AWSY test for how much memory that new tab page takes up, so we don't lose sight of its impact.That would also make regressions/improvements in the new tab page itself show up properly, where they might not currently meet thresholds for reporting on the "main" AWSY tests. Eric, does that sound sane? How hard would it be to get to that stage?

That sounds reasonable. Someone would have to add a separate test that loads the new tab page. That should be pretty simple, it's basically just the startup portion of the test. Then probably a front-end/marionette dev would need to modify the test_memory_usage test to somehow open a page in a new tab without first opening a new tab to a blank page within the constraints of the marionette framework.

I'm not sure I'm the best person to own doing this work, so if someone else wants to take it I'm all for it. I'm guessing the simplest way to make this work is:

0. disable new tab page preloading, force the privileged content process on;
1. create a test that loads the about:home/about:newtab page on startup "naturally", ie the same way a "normal" Firefox profile would by default
2. measure the privilege content process memory then
3. add 5 more tabs that also load about:newtab, measure that (after full pageload etc.)
4. close the tabs again, GC, measure again.

Mike/Eric, does that seem like it'd work? Mike, any tips from bug 1529965 on how to ensure the new tab page has fully loaded? Eric, should we measure the main process and privileged content process separately (should we even measure both?)

This seems reasonable. For this test I'd just focus on measuring the privileged content process. The base_memory_usage.py test is probably a good starting point. You can see that we filter to just the Web Content process there. Instead we'd just change that to Privileged Content. As far as timing goes I don't think we need to overthink this, just sleep for a bit.

In terms of being "done loading" we might want to consider loading a layout with simulated content (in the new Discovery Stream implementation, since that's what will be turned on in 68 and going forward) to get a more accurate measurement. Scott (:thecount) would be the best person to talk to about that since Pocket will be leading development on New Tab going forward.