Mapping a physical device to a pointer in User space

Question

We have an embedded system where a memory mapped device is connected, and an ARM CPU runs Linux. The device is located at address 0x40400000 and occupies a megabyte (most of it is not backed by an actual memory, but the address space is mapped to the device anyway). We currently don't have a device driver for this device.

In the device there is a special read-only register (called CID) at address 0x404f0704. This register contains the value CID = 0x404. I am trying to read this register from a program running on the ARM.

Searching the net I learned about the mmap() function that supposedly lets me access a physical address from userspace. So, trying to follow a couple of examples I found, I wrote the following test:

#include <sys/mman.h>
#include <fcntl.h>
#include <err.h>
#include <stdio.h>
#include <stdlib.h>

int main(void)
{
    void          *pdev = (void *) 0x40400000;
    size_t         ldev = (1024*1024);
    int           *pu;
    int  volatile *pcid;
    int  volatile  cid;

    pu = mmap(pdev, ldev, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
    if (pu == MAP_FAILED)
        errx(1, "mmap failure");

    pcid = (int *) (((void *) pu) + 0xf0704);

    printf("pu    = %08p\n", pu);
    printf("pcid  = %08p\n", pcid);

    cid = *pcid;
    printf("CID   = %x\n", cid);

    munmap(pu, ldev);

    return (EXIT_SUCCESS);
}

Compiling with the ARM cross-compiler:

a-gcc -O0 -g3 -o mmap-test.elf mmap-test.c

I can't get the expected result. What I see is that:

pu   = 0x40400000
pcid = 0x404f0704
CID  = 0

instead of the expected

CID  = 404

What am I missing / doing wrong here?

---

UPDATE:

I found another demo program and following its code I was able to get my code working:

int main(void)
{
    off_t          dev_base = 0x40400000;
    size_t         ldev = (1024 * 1024);
    unsigned long  mask = (1024 * 1024)-1;
    int           *pu;
    void          *mapped_base;
    void          *mapped_dev_base;
    int  volatile *pcid;
    int  volatile  cid;
    int            memfd;

    memfd = open("/dev/mem", O_RDWR | O_SYNC);
    mapped_base = mmap(0, MAP_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED, memfd, dev_base & ~MAP_MASK);
    if (mapped_base == MAP_FAILED)
        errx(1, "mmap failure");
    mapped_dev_base = mapped_base + (dev_base & MAP_MASK);
    pu = mapped_dev_base;

    pcid = (int *) (((void *) pu) + 0xf0704);

    printf("pu    = %08p\n", pu);
    printf("pcid  = %08p\n", pcid);

    cid = *pcid;
    printf("CID   = %x\n", cid);

    munmap(mapped_base, ldev);
    close(memfd);

    return (EXIT_SUCCESS);
}

Still, I am not so sure why the 1st version did not work. My understanding was that once you use MAP_ANONYMOUS you do not need a file handle for the mapping. Also, I obviously mistaken the addr argument (pepi in my 1st version) to be the physical address. If I am right now, then this is actually the virtual address.

Solution

Mmap is the function which usually works with virtual addresses. When you call mmap(... MAP_ANONYMOUS) (or mmap of /dev/zero file) it will give you some amount of new virtual memory, filled with zero. Address returned will be address of virtual memory.

You can mmap some file (without MAP_ANONYMOUS) and then mmap will map file contents into some virtual memory range.

The device is located at address 0x40400000

Device MMIO is located in Physical memory; any process can use virtual address 0x40400000; but they will be mapped (translated) to some free physical page by MMU (memory management unit). You can't just ask OS for some virtual memory and expect that is will be mmaped to device range (it will be variant of hell).

But there is a special device, /dev/mem, which can be used as File containing all physical memory. When you mmaps /dev/mem you are actually asking OS to create new mapping of some virtual memory into asked physical range.

In your invocation of mmap:

 mapped_base = mmap(0, MAP_SIZE, PROT_READ|PROT_WRITE, 
   MAP_SHARED, memfd, dev_base & ~MAP_MASK);

you ask to map physical memory range [0x40400000 .. 0x4050000-1] (one megabyte; not including byte 0x40500000) into some megabyte of virtual memory (its starting address is returned by mmap).