BTW, if you've ever wondered why I'm so preoccupied about the whole mrprotect story, I recommend reading my year old research on the topic which can be found here.
Well, it turns out to be, that with the release of the new Firefox things got much easier actually! There is no need to patch the source anymore, we just need to disable the 'jit' during source configuration. Not only this allows us to actually compile Firefox 5 on Gentoo hardened (sic!) but actually allows to run it with mprotect enabled too! But first things first...
An attempt to install Firefox 5 on Gentoo Hardened system is likely to end up with emerge failing and entry in kern.log similar to this:
grsec: denied RWX mmap of
by /var/tmp/portage/www-client/firefox-5.0/work/mozilla-release/obj-x86_64-unknown-linux-gnu/dist/bin/xpcshell[xpcshell:10891] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:10882] uid/euid:0/0 gid/egid:0/0
xpcshell: segfault at 41ea0ddc ip 00006b9475051ed4 sp 000078b37e81b6f0 error 4 in libxul.so[6b9474031000+1823000]
While jit seems to like RWX memory pages a lot, Gentoo hardened users do not...;)
But fear not! The trustworthy hack for Firefox 4 works here too, and we don't need to hack the source, it's just enough to add --disable-methodjit the configure script. (For more information on how to create your local overlay have a look at my previous post or refer to your favourite search engine ;) . So basically you could add this to the ebuild, somewhere in the src_configure() section:
if ! use jit ; then
mozconfig_annotate '' --disable-methodjit
..recompile...and off you go! At the moment the firefox binary (/usr/lib/firefox/firefox-bin) is quite likely to have mprotect automatically disabled so you might need to enable it by hand by running:
paxctl -M /usr/lib/firefox/firefox-bin
Bear in mind that Firefox now runs plugins in a separate process - plugin-container. It can also be mprotect enabled or disabled, so you might want to check it too...it's worth noting that enabling mprotect on plugin-container will make Firefox crash probably on every use of Flash or Java, but hey, it's secure then, is it not...? ;)
Gentoo bug tracking all this can be found here.