Repository navigation
high load with firefox 52.9.0esr 32bit on 64bit linuxmint #150
Description
Prerequisites
- I verified that this is not a filter issue
- Filter issues MUST be reported at filter issue tracker
- This is not a support issue or a question
- Support issues and questions are handled at /r/uBlockOrigin
- I performed a cursory search of the issue tracker to avoid opening a duplicate issue
- Your issue may already be reported.
- I tried to reproduce the issue when...
- uBlock Origin is the only extension
- uBlock Origin with default lists/settings
- using a new, unmodified browser profile
- I am running the latest version of uBlock Origin
- I checked the documentation to understand that the issue I report is not a normal behavior
Description
fresh firefox 52.9.0esr 32bit from mozilla on a nearly fresh linuxmint 18.3 64bit with 4GB RAM
starting firefox leads immediatly to high load and unusable firefox.
same profile with 64bit-firefox: no problem (but firefox needs more memory due to 64bit)
so maybe a bitsize-issue?
A specific URL where the issue occurs
any
Steps to Reproduce
- [First Step]
install 32bit firefox 52.9.0esr - [Second Step]
install ublock-origin - [and so on...]
start firefox, try to select menus or try to load a website, do anything and wait forever...
Expected behavior:
just works
Actual behavior:
up to 100% load (top) without any progress visible
Your environment
- uBlock Origin version: 1.16.14, but also tried earlier ones
- Browser Name and version: firefox 52.9.0esr 32bit
- Operating System and version: linuxmint 18.3 64bit xfce-gui
- System: i5-6200U with 4GB, at least 2GB available during tests after starting firefox (no out-of-memory-issue)
Activity
uBlock-user commentedon Jul 29, 2018
So a bad 32 bit firefox build possibly
uBlock-user commentedon Jul 29, 2018
You mean my bad 32 bit build claim ?
gorhill commentedon Jul 29, 2018
The WebAssembly code was an experiment to find out if this helped performance-wise. It did not but I kept the code around to revisit eventually to find out whether the overhead incurred when crossing the js/wasm barrier has been lowered.
gorhill commentedon Jul 29, 2018
Possibly an issue with running Firefox 32-bit in a 64-bit linux? (an unlikely scenario, hence why this wasn't reported before?)
I am running mint 64-bit, how do I get to install the missing dependencies?
gorhill commentedon Jul 29, 2018
Alright it's asking me to pull in a whole lot of other stuff, and frankly I rather not pollute my current installation for a case which appears to me as an issue specific with Firefox 32-bit running in a 64-bit environment. uBO is OS agnostic, it's just javascript, it does not care whether integers/pointers are 32- or 64-bit.
As per call stack, it spins forever in the
vaccuummethod, which just make use of plain javascript arrays, if there was an actual issue with uBO, it would be with all uBOs, this is a core code path always hit as soon as filters are loaded.sh-dvl commentedon Jul 29, 2018
first, thanks for your tests on sunday...
tested again:
I'll try it today or tomorrow on a pure 32bit-machine to verify, if it's really a problem of 32bit on 64bit-machine
12 remaining items
gorhill commentedon Jul 31, 2018
I will need to investigate before an issue is opened on bugzilla, no point dumping this in there until the issue is investigated. Now that I think about it I think I have a linux mint 32-bit version on a laptop I no longer use.
uBlock-user commentedon Jul 31, 2018
In regards to
TypeError: can't access dead object webbrowser.js:2508:1there was a bug filed in bugzilla, could this be related to that ?gorhill commentedon Jul 31, 2018
I can't remember the name of the setting now, but there is something in
about:configwhere we can adjust the timeout value for when javascript execute. I remember I had to advise someone a while ago to maybe increase this value because he was using old hardware and this was causing uBO to time out in some places (filter list parsing maybe, not sure), and on older version of Firefox, the timeout was cascading into more issues because Firefox was trying to show the prompter from the background script (which is what you see in the call stacks above).So my guess is that there is a performance issue with Firefox 32-bit + older hardware (?) causing this issue. The vacuum method can be expensive for some of the very long
domain=options in EasyList, and this may be worse for some older Firefox -- I always benchmarked using current Firefox. Though the vacuum method itself is known to be CPU-intensive, it's lazily called only once for a givendomain=option and once its done, the benefit is faster domain lookups when thedomain=option has to be evaluated afterward.gorhill commentedon Jul 31, 2018
@sh-dvl Something to experiment: what results do you obtain on that benchmark for the "Create large-size dictionary / Trie-based" entry? I get this on my side with Nitghtly:
"Trie-based" is essentially the
vacuummethod.sh-dvl commentedon Jul 31, 2018
can currently only try on pure 32bit-system, but for "Tries-based" I have to increase
dom.max_script_run_time, else firefox warn for slow-down and if not clickedwaitwithin some seconds, firefox seems to stop it without no further notice.after increase there still nothing happen after some minutes, but on java-console, there comes a
uncaught exception: out of memory (unbekannt)seeing this, I started a ps + a free within a loop, calling them every second, and near that OOM it says:
so there should be no OOM in firefox, but maybe firefox have a self-defined limit?
output until then with no addons active (even that won't really help you):
if you've suggestions, how to try this in another way, I'll try it, but for the next hours I can't due to other undertakings
gorhill commentedon Jul 31, 2018
I don't see the "Trie-based" in that output, that is the one which is of interest -- and especially the output for "large dictionary" (maybe the
vacuummethod also broke in that benchmark?)The "out of memory" condition in abnormal, this suggests something is breaking the algorithm. The
vacuummethod is actually rather simple, and I fail to see how it could break. One thing is I don't know how it would behave if it were fed a hostname with Unicode characters, that is not supposed to happen however, if there was such an issue surely this would also break in the 64-bit version.Anyway I will try to reproduce on that laptop once I get the time to do so.
gorhill commentedon Jul 31, 2018
Ok I could reproduce the issue with Firefox 52 ESR on Linux Mint 32-bit, just with laoding the benchmark and clicking "Creation" button. The issue occurs immediately when the "Trie-based" test is launched for the small dictionary. I will keep investigating.
gorhill commentedon Jul 31, 2018
I stepped into the code and all is working fine, i.e. as expected. However when let the benchmark run, I see memory going up fast in system monitor -- to the point where I have to abort the script or the OS will end up swap-paging, causing full desktop freeze etc. So something appears to be wrong with Firefox's 52 TypedArray on the 32-bit version -- possibly only a linux issue given that there is no widespread reports of the issue here. Will need to look at TypedArray issues fixed on bugzilla to find out if something was fixed which we can relate to the issue here.
gorhill commentedon Jul 31, 2018
To be sure nobody waste their time while I keep investigating, this has nothing to do with TypedArray.
gorhill commentedon Jul 31, 2018
At this point it does look like an issue with Firefox.
I added a
debuggerstatement to be taken whenever thevacuumseems to take longer than it should. It is being taken, and I can see that the input array, which is supposed to contain only integer numbers has values like1.11...e-311in it (might be one or more). The exact value is random but it's always something very small, likee-[160...312].I added code to detect such number at array filling time, in the
add()method, and such number is not being detected.I also added code to detect bad values in the array at the top of
vacuum(), and it's detected randomly. Sometime even when there is nothing bad found in the array at the top ofvacuum, something bad is found during processing of the array (the content of the input array is not modified during processing).This suggests that for some reasons the content of the array is randomly mutated internally after
add()and/or before or duringvacuum()and this causes the issue. At this point this is confirmed to be an issue specific to Firefox 52 ESR 32-bit on Linux.There is no need for uBO, just the benchmark is enough to trigger the issue.
sh-dvl commentedon Jul 31, 2018
well, many thanks to all of you, who helped to isolate possible cause of this problem.
this evening I had the chance to test on a single-user i7-8xxx with 32GB RAM (so no performance-issue should occure) with an 32bit 52.8.0esr-firefox: no problem. even the benchmark runs well.
using a 32bit 52.9.0esr, the same issue as described occures.
using a 64bit 52.9.0esr again no problem, even with the benchmark
I've now filed a bug at https://bugzilla.mozilla.org/show_bug.cgi?id=1479907