Repository navigation
high load with firefox 52.9.0esr 32bit on 64bit linuxmint #150
Description
Activity
same profile with 64bit-firefox: no problem
So a bad 32 bit firefox build possibly
Looks like you may be right.
You mean my bad 32 bit build claim ?
The WebAssembly code was an experiment to find out if this helped performance-wise. It did not but I kept the code around to revisit eventually to find out whether the overhead incurred when crossing the js/wasm barrier has been lowered.
Possibly an issue with running Firefox 32-bit in a 64-bit linux? (an unlikely scenario, hence why this wasn't reported before?)
I am running mint 64-bit, how do I get to install the missing dependencies?
Alright it's asking me to pull in a whole lot of other stuff, and frankly I rather not pollute my current installation for a case which appears to me as an issue specific with Firefox 32-bit running in a 64-bit environment. uBO is OS agnostic, it's just javascript, it does not care whether integers/pointers are 32- or 64-bit.
As per call stack, it spins forever in the
vaccuummethod, which just make use of plain javascript arrays, if there was an actual issue with uBO, it would be with all uBOs, this is a core code path always hit as soon as filters are loaded.first, thanks for your tests on sunday...
tested again:
- tried with all addons deactivated, all runs well, no error on stdout, following on jsconsole:
[Exception... "Component returned failure code: 0x80004004 (NS_ERROR_ABORT) [nsIPrefBranch.getComplexValue]" nsresult: "0x80004004 (NS_ERROR_ABORT)" location: "JS frame :: resource://app/components/nsBrowserContentHandler.js :: get startPage :: line 565" data: no] (unbekannt) [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIPrefBranch.getComplexValue]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: resource://app/components/nsBrowserContentHandler.js :: get startPage :: line 565" data: no] (unbekannt)- with only uBO active, again high load and bad response (10s or more) to open menu, addons-page or anything else. various outputs:
1532881733362 [email protected] WARN Loading extension '[email protected]': Reading manifest: Error processing permissions.1: Unknown permission "privacy" Extension error: TypeError: command.suggested_key is null chrome://browser/content/ext-commands.js 79 [[Exception stack loadCommandsFromManifest@chrome://browser/content/ext-commands.js:79:11 CommandList@chrome://browser/content/ext-commands.js:24:19 @chrome://browser/content/ext-commands.js:222:30 runSafeSyncWithoutClone@resource://gre/modules/ExtensionUtils.jsm:71:14 emit/promises<@resource://gre/modules/ExtensionUtils.jsm:384:55 emit@resource://gre/modules/ExtensionUtils.jsm:383:20 runManifest@resource://gre/modules/Extension.jsm:733:23 startup/<@resource://gre/modules/Extension.jsm:813:14 Current stack runSafeSyncWithoutClone@resource://gre/modules/ExtensionUtils.jsm:73:129 emit/promises<@resource://gre/modules/ExtensionUtils.jsm:384:55 emit@resource://gre/modules/ExtensionUtils.jsm:383:20 runManifest@resource://gre/modules/Extension.jsm:733:23 startup/<@resource://gre/modules/Extension.jsm:813:14 ]] Extension error: out of memory undefined undefined [[Exception stack undefinedCurrent stack runSafeSyncWithoutClone@resource://gre/modules/ExtensionUtils.jsm:73:129 runSafeSync@resource://gre/modules/ExtensionUtils.jsm:104:37 runSafe@resource://gre/modules/ExtensionCommon.jsm:123:32 receiveMessage@resource://gre/modules/ExtensionChild.jsm:586:49 _callHandlers/<@resource://gre/modules/MessageChannel.jsm:574:17 _callHandlers@resource://gre/modules/MessageChannel.jsm:573:14 _handleMessage/deferred.promise<@resource://gre/modules/MessageChannel.jsm:637:7 _handleMessage@resource://gre/modules/MessageChannel.jsm:634:24 receiveMessage@resource://gre/modules/MessageChannel.jsm:165:5 ]][Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIPrefBranch.getComplexValue]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: resource://app/components/nsBrowserContentHandler.js :: get startPage :: line 565" data: no] (unbekannt) Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 Error: Invalid option requestBody WebRequest.jsm:88:15 TypeError: commandsMap.get(...) is undefined[Weitere Informationen] ext-commands.js:252:9 Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 [Exception... "Component returned failure code: 0x80004004 (NS_ERROR_ABORT) [nsIPrefBranch.getComplexValue]" nsresult: "0x80004004 (NS_ERROR_ABORT)" location: "JS frame :: resource://app/components/nsBrowserContentHandler.js :: get startPage :: line 565" data: no] (unbekannt) 1532883189548 [email protected] WARN Loading extension '[email protected]': Reading manifest: Error processing permissions.1: Unknown permission "privacy" command.suggested_key is null ext-commands.js:79 [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIPrefBranch.getComplexValue]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: resource://app/components/nsBrowserContentHandler.js :: get startPage :: line 565" data: no] (unbekannt) Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 Error: Invalid option requestBody WebRequest.jsm:88:15 TypeError: commandsMap.get(...) is undefined[Weitere Informationen] ext-commands.js:252:9 Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 Error: Invalid option requestBody WebRequest.jsm:88:15 TypeError: commandsMap.get(...) is undefined[Weitere Informationen] ext-commands.js:252:9 Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 [Exception... "Component returned failure code: 0x80004004 (NS_ERROR_ABORT) [nsIPrefBranch.getComplexValue]" nsresult: "0x80004004 (NS_ERROR_ABORT)" location: "JS frame :: resource://app/components/nsBrowserContentHandler.js :: get startPage :: line 565" data: no] (unbekannt) 1532883302460 [email protected] WARN Loading extension '[email protected]': Reading manifest: Error processing permissions.1: Unknown permission "privacy" command.suggested_key is null ext-commands.js:79 [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIPrefBranch.getComplexValue]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: resource://app/components/nsBrowserContentHandler.js :: get startPage :: line 565" data: no] (unbekannt) Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 Error: Invalid option requestBody WebRequest.jsm:88:15 TypeError: commandsMap.get(...) is undefined[Weitere Informationen] ext-commands.js:252:9 Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDOMWindowUtils.isParentWindowMainWidgetVisible] nsPrompter.js:350 Error: Script terminated by timeout at: HNTrieBuilder.prototype.vacuum@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/hntrie.js:333:13 HNTrieBuilder.fromDomainOpt@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/hntrie.js:85:12 FilterOriginHitSet.prototype<.matchOrigin.value@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:890:30 FilterOrigin.prototype.match@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:794:12 FilterBucket.prototype.match@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:1319:14 FilterContainer.prototype.matchTokens@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:2536:29 FilterContainer.prototype.matchStringGenericHide@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:2564:21 FilterContainer.prototype.matchStringExactType@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:2588:16 PageStore.prototype.init@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/pagestore.js:322:22 PageStore.prototype.reuse@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/pagestore.js:378:5 µb.bindTabToPageStats@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/tab.js:878:5 vAPI.tabs.onNavigation@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/tab.js:491:25 vAPI.tabs.registerListeners/onCommitted@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/vapi-background.js:364:9 hntrie.js:333:13 Invalid match pattern: 'moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/web_accessible_resources/*' MatchPattern.jsm:52 Invalid match pattern: 'ws://*/*' MatchPattern.jsm:52 Invalid match pattern: 'wss://*/*' MatchPattern.jsm:52 NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDOMWindowUtils.isParentWindowMainWidgetVisible] nsPrompter.js:350 Error: Script terminated by timeout at: HNTrieBuilder.prototype.vacuum@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/hntrie.js:333:13 HNTrieBuilder.fromDomainOpt@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/hntrie.js:85:12 FilterOriginHitSet.prototype<.matchOrigin.value@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:890:30 FilterOrigin.prototype.match@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:794:12 FilterContainer.prototype.matchTokens@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:2527:33 FilterContainer.prototype.matchString@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/static-net-filtering.js:2765:22 PageStore.prototype.filterRequest@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/pagestore.js:640:18 onBeforeRequest@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/traffic.js:157:18 vAPI.net.registerListeners/onBeforeRequest@moz-extension://f5dc6456-d045-49f2-bfa8-f105cf573a48/js/vapi-webrequest.js:200:16 hntrie.js:333:131532883302460 [email protected] WARN Loading extension '[email protected]': Reading manifest: Error processing permissions.1: Unknown permission "privacy" Extension error: TypeError: command.suggested_key is null chrome://browser/content/ext-commands.js 79 [[Exception stack loadCommandsFromManifest@chrome://browser/content/ext-commands.js:79:11 CommandList@chrome://browser/content/ext-commands.js:24:19 @chrome://browser/content/ext-commands.js:222:30 runSafeSyncWithoutClone@resource://gre/modules/ExtensionUtils.jsm:71:14 emit/promises<@resource://gre/modules/ExtensionUtils.jsm:384:55 emit@resource://gre/modules/ExtensionUtils.jsm:383:20 runManifest@resource://gre/modules/Extension.jsm:733:23 startup/<@resource://gre/modules/Extension.jsm:813:14 Current stack runSafeSyncWithoutClone@resource://gre/modules/ExtensionUtils.jsm:73:129 emit/promises<@resource://gre/modules/ExtensionUtils.jsm:384:55 emit@resource://gre/modules/ExtensionUtils.jsm:383:20 runManifest@resource://gre/modules/Extension.jsm:733:23 startup/<@resource://gre/modules/Extension.jsm:813:14 ]] ExceptionHandler::GenerateDump cloned child 2601 ExceptionHandler::WaitForContinueSignal waiting for continue signal... ExceptionHandler::SendContinueSignalToChild sent continue signal to child ###!!! [Parent][RunMessage] Error: Channel error: cannot send/recv ###!!! [Parent][RunMessage] Error: Channel error: cannot send/recv ###!!! [Parent][RunMessage] Error: Channel error: cannot send/recv ###!!! [Parent][RunMessage] Error: Channel error: cannot send/recv ###!!! [Parent][RunMessage] Error: Channel error: cannot send/recvI'll try it today or tomorrow on a pure 32bit-machine to verify, if it's really a problem of 32bit on 64bit-machine
12 remaining items
I will need to investigate before an issue is opened on bugzilla, no point dumping this in there until the issue is investigated. Now that I think about it I think I have a linux mint 32-bit version on a laptop I no longer use.
In regards to
TypeError: can't access dead object webbrowser.js:2508:1there was a bug filed in bugzilla, could this be related to that ?I can't remember the name of the setting now, but there is something in
about:configwhere we can adjust the timeout value for when javascript execute. I remember I had to advise someone a while ago to maybe increase this value because he was using old hardware and this was causing uBO to time out in some places (filter list parsing maybe, not sure), and on older version of Firefox, the timeout was cascading into more issues because Firefox was trying to show the prompter from the background script (which is what you see in the call stacks above).So my guess is that there is a performance issue with Firefox 32-bit + older hardware (?) causing this issue. The vacuum method can be expensive for some of the very long
domain=options in EasyList, and this may be worse for some older Firefox -- I always benchmarked using current Firefox. Though the vacuum method itself is known to be CPU-intensive, it's lazily called only once for a givendomain=option and once its done, the benefit is faster domain lookups when thedomain=option has to be evaluated afterward.@sh-dvl Something to experiment: what results do you obtain on that benchmark for the "Create large-size dictionary / Trie-based" entry? I get this on my side with Nitghtly:
Create large-size dictionary - Set-based x 7,076 ops/sec ±5.36% (47 runs sampled) - Regex-based x 9,944 ops/sec ±3.69% (53 runs sampled) - Trie-based x 1,312 ops/sec ±2.58% (58 runs sampled) Done."Trie-based" is essentially the
vacuummethod.can currently only try on pure 32bit-system, but for "Tries-based" I have to increase
dom.max_script_run_time, else firefox warn for slow-down and if not clickedwaitwithin some seconds, firefox seems to stop it without no further notice.after increase there still nothing happen after some minutes, but on java-console, there comes a
uncaught exception: out of memory (unbekannt)seeing this, I started a ps + a free within a loop, calling them every second, and near that OOM it says:
total used free shared buff/cache available Mem: 2062324 381492 1404868 46576 275964 1402696 Swap: 4193276 545980 3647296 0 1000 20071 16925 20 0 808068 224048 poll_s Sl+ pts/2 0:03 /opt/firefox/firefox -jsconsole 0 1000 20114 20071 20 0 734932 102188 poll_s Sl+ pts/2 1:19 /opt/firefox-52.9.0esr/plugin-container -greomni /opt/firefox-52.9.0esr/omni.ja -appomni /opt/firefox-52.9.0esr/browser/omni.ja -appdir /opt/firefox-52.9.0esr/browser 20071 true tabso there should be no OOM in firefox, but maybe firefox have a self-defined limit?
output until then with no addons active (even that won't really help you):
Benchmarking, the higher ops/sec the better. Firefox 52.0 on Linux i686. Create small-size dictionary - Set-based x 811,852 ops/sec ±7.28% (39 runs sampled) - Regex-based x 443,424 ops/sec ±3.76% (39 runs sampled)if you've suggestions, how to try this in another way, I'll try it, but for the next hours I can't due to other undertakings
output until then with no addons active (even that won't really help you):
I don't see the "Trie-based" in that output, that is the one which is of interest -- and especially the output for "large dictionary" (maybe the
vacuummethod also broke in that benchmark?)The "out of memory" condition in abnormal, this suggests something is breaking the algorithm. The
vacuummethod is actually rather simple, and I fail to see how it could break. One thing is I don't know how it would behave if it were fed a hostname with Unicode characters, that is not supposed to happen however, if there was such an issue surely this would also break in the 64-bit version.Anyway I will try to reproduce on that laptop once I get the time to do so.
Ok I could reproduce the issue with Firefox 52 ESR on Linux Mint 32-bit, just with laoding the benchmark and clicking "Creation" button. The issue occurs immediately when the "Trie-based" test is launched for the small dictionary. I will keep investigating.
I stepped into the code and all is working fine, i.e. as expected. However when let the benchmark run, I see memory going up fast in system monitor -- to the point where I have to abort the script or the OS will end up swap-paging, causing full desktop freeze etc. So something appears to be wrong with Firefox's 52 TypedArray on the 32-bit version -- possibly only a linux issue given that there is no widespread reports of the issue here. Will need to look at TypedArray issues fixed on bugzilla to find out if something was fixed which we can relate to the issue here.
To be sure nobody waste their time while I keep investigating, this has nothing to do with TypedArray.
At this point it does look like an issue with Firefox.
I added a
debuggerstatement to be taken whenever thevacuumseems to take longer than it should. It is being taken, and I can see that the input array, which is supposed to contain only integer numbers has values like1.11...e-311in it (might be one or more). The exact value is random but it's always something very small, likee-[160...312].I added code to detect such number at array filling time, in the
add()method, and such number is not being detected.I also added code to detect bad values in the array at the top of
vacuum(), and it's detected randomly. Sometime even when there is nothing bad found in the array at the top ofvacuum, something bad is found during processing of the array (the content of the input array is not modified during processing).This suggests that for some reasons the content of the array is randomly mutated internally after
add()and/or before or duringvacuum()and this causes the issue. At this point this is confirmed to be an issue specific to Firefox 52 ESR 32-bit on Linux.There is no need for uBO, just the benchmark is enough to trigger the issue.
well, many thanks to all of you, who helped to isolate possible cause of this problem.
this evening I had the chance to test on a single-user i7-8xxx with 32GB RAM (so no performance-issue should occure) with an 32bit 52.8.0esr-firefox: no problem. even the benchmark runs well.
using a 32bit 52.9.0esr, the same issue as described occures.
using a 64bit 52.9.0esr again no problem, even with the benchmark
I've now filed a bug at https://bugzilla.mozilla.org/show_bug.cgi?id=1479907
Prerequisites
Description
fresh firefox 52.9.0esr 32bit from mozilla on a nearly fresh linuxmint 18.3 64bit with 4GB RAM
starting firefox leads immediatly to high load and unusable firefox.
same profile with 64bit-firefox: no problem (but firefox needs more memory due to 64bit)
so maybe a bitsize-issue?
A specific URL where the issue occurs
any
Steps to Reproduce
install 32bit firefox 52.9.0esr
install ublock-origin
start firefox, try to select menus or try to load a website, do anything and wait forever...
Expected behavior:
just works
Actual behavior:
up to 100% load (top) without any progress visible
Your environment