How to get memory block length after malloc?

Question

I thought that I couldn't retrieve the length of an allocated memory block like the simple .length function in Java. However, I now know that when malloc() allocates the block, it allocates extra bytes to hold an integer containing the size of the block. This integer is located at the beginning of the block; the address actually returned to the caller points to the location just past this length value. The problem is, I can't access that address to retrieve the block length.

#include <stdlib.h>
#include <stdio.h>
int main(void)
{
        char *str;
        str = (char*) malloc(sizeof(char)*1000);
        int *length;
        length = str-4; /*because on 32 bit system, an int is 4 bytes long*/
        printf("Length of str:%d\n", *length);
        free(str);
}

**Edit: I finally did it. The problem is, it keeps giving 0 as the length instead of the size on my system is because my Ubuntu is 64 bit. I changed str-4 to str-8, and it works now.

If I change the size to 2000, it produces 2017 as the length. However, when I change to 3000, it gives 3009. I am using GCC.

Solution 1

What you're doing is definitely wrong. While it's almost certain that the word just before the allocated block is related to the size, even so it probably contains some additional flags or information in the unused bits. Depending on the implementation, this data might even be in the high bits, which would cause you to read the entirely wrong length. Also it's possible that small allocations (e.g. 1 to 32 bytes) are packed into special small-block pages with no headers, in which case the word before the allocated block is just part of another block and has no meaning whatsoever in relation to the size of the block you're examining.

Just stop this misguided and dangerous pursuit. If you need to know the size of a block obtained by malloc, you're doing something wrong.

Solution 2

You don't have to track it by your self!

size_t malloc_usable_size (void *ptr);

But it returns the real size of the allocated memory block! Not the size you passed to malloc!

Solution 3

I would suggest you create your own malloc wrapper by compiling and linking a file which defines my_malloc() and then overwiting the default as follows:

// my_malloc.c

#define malloc(sz) my_malloc(sz)

typedef struct {
    size_t size;
} Metadata;

void *my_malloc(size_t sz) {
    size_t size_with_header = sz + sizeof(Metadata);
    void* pointer = malloc(size_with_header);

    // cast the header into a Metadata struct
    Metadata* header = (Metadata*)pointer;
    header->size = sz;    
    // return the address starting after the header 
    // since this is what the user needs
    return pointer + sizeof(Metadata);
}

then you can always retrieve the size allocated by subtracting sizeof(Metadata), casting that pointer to Metadata and doing metadata->size:

Metadata* header = (Metadata*)(ptr - sizeof(Metadata));
printf("Size allocated is:%lu", header->size); // don't quote me on the %lu ;-)

Solution 4

You're not supposed to do that. If you want to know how much memory you've allocated, you need to keep track of it yourself.

Looking outside the block of memory returned to you (before the pointer returned by malloc, or after that pointer + the number of bytes you asked for) will result in undefined behavior. It might work in practice for a given malloc implementation, but it's not a good idea to depend on that.

Solution 5

This is not Standard C. However, it is supposed to work on Windows operatings systems and might to be available on other operating systems such as Linux (msize?) or Mac (alloc_size?), as well.

size_t _msize( void *memblock );

_msize() returns the size of a memory block allocated in the heap.

See this link: http://msdn.microsoft.com/en-us/library/z2s077bc.aspx

View more solutions

Comments

[Rafe Kettler]

@Amumu how you can do so depends on the compiler and the architecture.

[metamatt]

It depends more on the implementation of malloc() itself, not really the compiler.

[R.. GitHub STOP HELPING ICE]

Not necessarily. See my answer.

[Amumu]

I see. Thanks. That's why it has different value for each size. I just try it to see how it works, and I won't do this in real things. However, there's an exercise in a book require to implement malloc() and free() based on its explained implementation, so I just wanna give it a try.

[R.. GitHub STOP HELPING ICE]

Cool, +1 for doing this as part of experimentation to "figure out how it works and try to implement" rather than for some horrible hackery.

[Amumu]

Do you know the way to filter out all the additional flags it append to the block, so I get the pure memory size? However, by guessing, I think I can't alter that, and have to altered my own way then. Also, I don't know why the sizeof(int) gives 4, while my system is 64 bits? Is there a way to let sizeof know my 64 bits system?

[R.. GitHub STOP HELPING ICE]

sizeof(int) is always 4 on the vast majority of unix systems, and probably always will be. Use long or better yet size_t if you want the system word size.

[SwiftMango]

@Amumu Nothing is dangerous if you know what you are doing, and the original poster is not misguided. It depends on the implementation. If you look at the implementation in the book of C programming language by Brian Kernighan, there is a general version of malloc and free. I am not sure it is the same version for GCC (most likely not). You can look into GCC's code, and see where they put the Header, and write your own function of getting the header info. It will not be a portable solution, but if you just use it yourself on the same compiler, it would be fine.

[SwiftMango]

This is a good one workaround than looking into the gcc source code.

[R.. GitHub STOP HELPING ICE]

@texasbruce: It is inherently dangerous and wrong unless you're programming for a specific version of the specific C implementation (library and compiler) you're going to be using, and nothing else. And even still, making that assumption is dangerous and wrong because it's likely that critical bugs will later be found in the implementation you're using, and that you'll need to upgrade, and if the assumptions you made based on the old version are no longer true with the new version, your program will break catastrophically.

[Thomas]

Worth noting that the memory returned by your wrapper will no longer be aligned on most architectures. I should know, I wrote the same and my code relying on alignment blew up in my face.

[Eloff]

I bridle when people have the lack of imagination to say there is no possible reason to justify doing something. There's always a plethora of valid reasons. In this case it's even safe on a wide variety of platforms. See malloc_usable_size() on linux and _msize() on windows.

[R.. GitHub STOP HELPING ICE]

@Eloff: I suppose you never follow the glibc bug tracker. There were critical bugs in malloc_usable_size that made it not safe, but dangerous to use, under certain configurations: sourceware.org/bugzilla/show_bug.cgi?id=1349 bugzilla.redhat.com/show_bug.cgi?id=576431 code.google.com/p/skipfish/issues/detail?id=61 ...

[Mike M]

That makes sense and this code should only be used for debug purposes under strict control conditions.

[Michael Graczyk]

FYI pointer arithmetic with void* is not allowed so this code will fail to compile on compliant compiles (for instance, GCC with -pedantic).

[Degustaf]

It's important to note that this function is not part of the C standard, and is a gnulib extension

[CAMOBAP]

Not only gnulib extension

[Fernando Costa Bertoldi]

@Thomas To fix alignment you could use a union with a member of type max_align_t: typedef union { size_t size; max_align_t ma; } Metadata;

[Ilian Zapryanov]

Interesting that in my linux (OpenSUSE 64) the code when substracting from int pointer to get the header with sizeof(Metadata) always returns me 0 size. But substracting with - 2 works.

[S.S. Anne]

@Hydroper Definitely not. That detects GCC, not the GNU C library. Use __GNU_LIBRARY__ instead.