Install 64 bit programs on a 32 bit OS with a 64 bit processor

Question

I'm curious. Is it possible to install a 64 bit program on a 32 bit OS with a 64 bit processor?

I'm running Linux on a raspberry pi 3 and I try to install a newer version of MongoDB:

armv7l GNU/Linux
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=raspbian
ID_LIKE=debian

Solution 1

Is it possible to install a 64 bit program on a 32 bit OS with a 64 bit processor?

In principle yes, but the processor and the OS have to support it.

On ARMv8, a 32-bit (Aarch32) kernel cannot run 64-bit (Aarch64) processes. This is a limitation of the processor.

There are other processors that don't have this limitation, for example it is possible to run x86_64 processes on top of an x86_32 kernel on an x86_64 processor, but few kernels support it, presumably because it's of limited utility (mostly, you save a bit of RAM in the kernel by making it 32-bit). Linux doesn't support it, but Solaris does.

You can keep your existing 32-bit OS if you run a 64-bit kernel. An Aarch64 Linux kernel can run Aarch32 processes. Raspbian doesn't support this out of the box, so you'd need to maintain both a 32-bit OS and a 64-bit OS. You can use either one as the main OS (i.e. the one that runs init and system services) and the other to run a specific program using chroot. See How do I run 32-bit programs on a 64-bit Debian/Ubuntu? for a practical approach.

Note that you will need to install all the libraries that the 64-bit program requires. Any given process must be either wholly 32-bit or wholly 64-bit, so you can't use a 32-bit library in a 64-bit executable.

Unless you have strong reasons to keep a 32-bit system, if you need to run a 64-bit executable, it would be easier to install a 64-bit system.

Note that the only thing that 64-bit programs can do but 32-bit programs can't is address more than about 3GB of virtual memory, which is of limited utility on a system with 1GB of RAM. You may get performance benefits from the extra, larger registers, but you'll also lose performance from the extra memory accesses.

Solution 2

On some architectures, yes. But not on ARM or x86.

You could use QEMU to emulate a 64-bit system, but you don't want to.

Solution 3

Upgrade only your kernel to a 64-bit one, so you will be able to run 64-bit binaries. Essentially, it will run your whole distribution in 32-bit compat mode, and your only 64-bit mongodb will be its normal mode.

But it doesn't deserve its price. Better to switch your mongodb to 32 bit. However, in this case there is a limitation, that your database cannot be bigger as 2GB, as it directly maps the whole thingy in virtual memory. If your db is bigger, only the kernel upgrade remains. (Thanks @duskwuff the extension!)

Btw, if your db doesn't want a very big load, or you can use some caching solution before it (for example: another, but 32bit mongo), then a cpu emulation could work. For that, start a googling for "qemu qemu-system-x86_64". Although such a solution would have likely an infeasible work need and it could be considered weird in productive environment.

In your place, I would use 32 bit mongo if it is for my db enough, or a 64 bit kernel if it is not.

Comments

[Admin]

Consider using a 64-bit OS instead. Raspbian is way behind the times; 64-bit Fedora is already available for the RPi3.

[Admin]

The problem is that the CPU is started (by the kernel) in 32-bit mode, so it looks like an old 32-bit machine. 64-bit registers and instructions are not available.

[Admin]

Right, which is why you’d need a 64-bit kernel (which Thomas mentioned).

[Admin]

Thanks for the nice explanation. So the solution for me would be to install another OS on my RPI3?

[crellee]

Would it be a disaster to use an emulator like QEMU to install and run a mongo database?

[Admin]

I just updated my answer.

[Admin]

@Thomas, some distros do support combined 32-/64-bit operation, starting with the 32-bit variant. Debian is one example, at least on x86: you can install the i386 variant, and that includes a 64-bit kernel, which allows 64-bit libraries and binaries to be installed too. The ARM support in Debian doesn’t allow this on ARM systems though (but you can install combined arm64 and armhf, if you start with arm64).

[Admin]

@StephenKitt I just added a link to the official arm64 port for Debian on the RPI3

[Admin]

@Thomas, saw that, thanks (I was mainly commenting on the first paragraph in your answer, since some Debian setups make this quite simple). Welcome to Unix.SE BTW!

[Admin]

So by installing ArchLinux ARM I overwrite my old OS and install a new? Or do I just replace some files on my RPI3 that makes it able for me to install 64bits software?

[Admin]

If you want install ArchLinux ARM you'll need to wipe your SD card (and erase everything on it), so make sure to have a backup of all your important files. @StephenKitt thank for the welcome :)

[Gilles 'SO- stop being evil']

You can run a 64-bit program on top of a 32-bit kernel on x86 if the kernel supports it. Linux doesn't, but Solaris does. On arm it isn't possible.

[Wildcard]

I would love any starting points for in-depth reading on just what is there in the 64 bit ARM processor (or what isn't there) which differs from x86 and thereby somehow prevents a 32-bit kernel developer who is coding for ARM from supporting the running of 64-bit processes. I find that difficult to imagine. (Not doubting, just very curious.)

[phyrfox]

@Wildcard developer.arm.com/products/architecture/a-profile/docs/den00‌​24/… "An AArch32 operating system cannot host a 64-bit application." There's more reading there than one would put in a comment, but basically, it's in the documentation.

[crellee]

Makes sense, but how would you switch your mongodb to 32bit?

[peterh]

@crellee apt-get install mongodb:i386 or some similar?

[Gilles 'SO- stop being evil']

@Wildcard In a nutshell, at any given time, either the processor is in 32-bit mode (“Aarch32 execution state”) and expecting Aarch32 instructions, or in 64-bit mode expecting Aarch64 instructions. The only way to switch modes is to switch between process and kernel (or kernel and hypervisor, or hypervisor and monitor). Switching to a lower-privilege mode can't switch from 32-bit to 64-bit, and switching to a higher-privilege mode always restores the previous mode. So it's impossible to arrange to have a 32-bit code running at a higher privilege than 64-bit code.

[Gilles 'SO- stop being evil']

@Wildcard (cont.) Presumably the reason for this restriction is that there's quite a lot of Aarch64-specific processor state and that Aarch32 code can't access. For example an Aarch32 kernel would be unable to save the registers of an Aarch64 process.

[crellee]

that will return a 'Unable to locate package mongodb:i386' error

[crellee]

Thanks for the answer. Based on what you said I think that I will try to install a 64bit OS.

[Admin]

Thank you very much for the help. I will try soon to install ArchLinux instead of my current Raspian.

[Wildcard]

@Gilles, thanks! But I don't see why that's CPU-dependent; why wouldn't it likewise be impossible for a 32-bit Solaris kernel to access the 64-bit registers of an x86 processor and thus impossible to context switch the 64 bit processes?

[Gilles 'SO- stop being evil']

@Wildcard I don't know enough about the x86 architecture to answer this.

[Admin]

Running a 32-bit MongoDB is a bad idea. The database is memory-mapped, so running it on a 32-bit system limits you to <2GB of data.

[crellee]

Yes, and you are limited to version: 2.4.14, which has a lot of limitations.

[peterh]

@duskwuff Right. But he is running it on a raspberry pi, I see a reasonable chance that his db won't be very big. I extended the post.

[peterh]

@crellee ..."or some similar"...

[crellee]

All right. But I’m still in need MongoDB functions like $lookup and geospatial queries - and for that I will be in need of a newer version of MongoDB than 2.4

[peterh]

@crellee Check this. There is mongodb 3.2 on i386. You don't need to use upstream binaries, it is open source. apt-get build-dep mongodb, apt-source mongodb, debian/rules build, debian/rules binary are your commands for a full recompilation on any architecture.

[Serge]

@Wildcard, its because of architectural decisions taken by particular development teams and time of these decisions. Intel's developers decided to add address- and data-size prefixes to the instruction set which let to temporarily switch the addressing mode or data size of an instruction. This was done in order to allow an early adoption of 64 bit capabilities of CPUs being developed at the time when there were no operating systems supporting 64bit data or addressing. In contrast, ARM developed their 64 bit specs when there were a plenty of OSes already supporting 64bit data and addressing.

[Iwillnotexist Idonotexist]

@crellee, Gilles: You don't need to look at exotic OSes to find examples of 32-bit kernels with 64-bit userlands. The extremely popular Mac OS X 10.4 "Tiger", 10.5 "Leopard" and 10.6 "Snow Leopard" kernels shipped in the K32 configuration for almost all Macs, save for a few server machines that were allowed to boot Snow Leopard's K64, but all of them were capable of running 64-bit userland processes (with progressively fewer restrictions).

[peterh]

@IgnacioVazquez-Abrams Try this. It is slow, but not catastrophically. The main reason is that qemu does cpu emulation only in the user space, the kernel calls (i.e. io waiting times) remain the same. On home systems, I like to use arm or mips binaries for some tools, just for fun :-) Or, sometimes, if there is not a very big cpu load, some security-critical, but not cpu-intensive tools could use some exotic architecture, to decrease the probability of buffer overrun attacks.

[Peter Cordes]

@Serge: What you describe is true for 286 -> 386: you can use 32-bit operand size in 16-bit real mode with prefixes, where the default operand size is 16. But Intel didn't even develop x86-64; that was AMD (which is why it's still sometimes called AMD64). And no, you can't use 64-bit operand-size in 32-bit mode, at all. REX prefixes repurpose the one-byte inc/dec register opcodes (0x40 .. 0x4F). In long mode (64-bit mode), the default operand size is 32, but the default address size is 64.

[Peter Cordes]

@Wildcard: IDK if compat-mode kernel / long-mode userspace was a design consideration at all. To save integer register state for context switches, Solaris (and OS X) must still be in long mode to access r8-r15, and the upper halves of rax-rsi. IDK if xsave / xrstor in compat mode can save the full vector state, either. So it's certainly not well supported or explicitly catered for. Probably the kernel entry points run in 64-bit (long) mode, and switch to 32-bit (compat) mode before jumping to the rest of the kernel. (x86 mode switch just takes a far jmp and doesn't affect regs.)

[crellee]

package architecture (i386) does not match system (armhf). So I will still be in need of changing my OS?

[peterh]

@crellee I've shown the i386 version to prove, that yes, there is mongodb for 32-bit architectures. However, I end the talk now with you, because 1) you are a help vampire 2) your under-education makes the communication disturbing for me. So, here is the end.

[Toby Speight]

Re the last paragraph - 64-bit addressing might be useful if you want to mmap() large files (which you're accessing in a more localised fashion) or if you're sharing binaries (e.g. over NFS).

[Serge]

@PeterCordes, my bad, you are correct. But Intel did not change the major direction taken long before (when it first introduced 32 bit processors): the kernel (supervisor, whatever is executed in ring 0) could create a tss of any kind regardless of its own -bit size'. In other words, it is possible to control 64-bit task from 16-bit protected mode kernel.