What's the best way to put a c-struct in an NSArray?

Question

What's the usual way to store c-structures in an NSArray? Advantages, disadvantages, memory handling?

Notably, what's the difference between valueWithBytes and valueWithPointer -- raised by justin and catfish below.

Here's a link to Apple's discussion of valueWithBytes:objCType: for future readers...

For some lateral thinking and looking more at performance, Evgen has raised the issue of using STL::vector in C++.

(That raises an interesting issue: is there a fast c library, not unlike STL::vector but much much lighter, that allows for the minimal "tidy handling of arrays" ...?)

So the original question...

For example:

typedef struct _Megapoint {
    float   w,x,y,z;
} Megapoint;

So: what's the normal, best, idiomatic way to store one's own structure like that in an NSArray, and how do you handle memory in that idiom?

Please note that I am specifically looking for the usual idiom to store structs. Of course, one could avoid the issue by making a new little class. However I want to know how the usual idiom for actually putting structs in an array, thanks.

BTW here's the NSData approach which is perhaps? not best...

Megapoint p;
NSArray *a = [NSArray arrayWithObjects:
    [NSData dataWithBytes:&p length:sizeof(Megapoint)],
    [NSData dataWithBytes:&p length:sizeof(Megapoint)],
    [NSData dataWithBytes:&p length:sizeof(Megapoint)],
        nil];

BTW as a point of reference and thanks to Jarret Hardie, here's how to store CGPoints and similar in an NSArray:

NSArray *points = [NSArray arrayWithObjects:
        [NSValue valueWithCGPoint:CGPointMake(6.9, 6.9)],
        [NSValue valueWithCGPoint:CGPointMake(6.9, 6.9)],
        nil];

(see How can I add CGPoint objects to an NSArray the easy way?)

Solution 1

NSValue doesn't only support CoreGraphics structures – you can use it for your own too. I would recommend doing so, as the class is probably lighter weight than NSData for simple data structures.

Simply use an expression like the following:

[NSValue valueWithBytes:&p objCType:@encode(Megapoint)];

And to get the value back out:

Megapoint p;
[value getValue:&p];

Solution 2

I would suggest you stick to the NSValue route, but if you really do wish to store plain 'ol struct datatypes in your NSArray (and other collection objects in Cocoa), you can do so -- albeit indirectly, using Core Foundation and toll-free bridging.

CFArrayRef (and its mutable counterpart, CFMutableArrayRef) afford the developer more flexibility when creating an array object. See the fourth argument of the designated initialiser:

CFArrayRef CFArrayCreate (
    CFAllocatorRef allocator,
    const void **values,
    CFIndex numValues,
    const CFArrayCallBacks *callBacks
);

This allows you to request that the CFArrayRef object use Core Foundation's memory management routines, none at all or even your own memory management routines.

Obligatory example:

// One would pass &kCFTypeArrayCallBacks (in lieu of NULL) if using CF types.
CFMutableArrayRef arrayRef = CFArrayCreateMutable(kCFAllocatorDefault, 0, NULL);
NSMutableArray *array = (NSMutableArray *)arrayRef;

struct {int member;} myStruct = {.member = 42};
// Casting to "id" to avoid compiler warning
[array addObject:(id)&myStruct];

// Hurray!
struct {int member;} *mySameStruct = [array objectAtIndex:0];

The above example completely ignores the issues with respect to memory management. The structure myStruct is created on the stack and hence is destroyed when the function ends -- the array will contain a pointer to an object that is no longer there. You can work around this by using your own memory management routines -- hence why the option is provided to you -- but then you have to do the hard work of reference counting, allocating memory, deallocating it and so on.

I would not recommend this solution, but will keep it here in case it is of interest to anyone else. :-)

---

Using your structure as allocated on the heap (in lieu of the stack) is demonstrated here:

typedef struct {
    float w, x, y, z;
} Megapoint;

// One would pass &kCFTypeArrayCallBacks (in lieu of NULL) if using CF types.
CFMutableArrayRef arrayRef = CFArrayCreateMutable(kCFAllocatorDefault, 0, NULL);
NSMutableArray *array = (NSMutableArray *)arrayRef;

Megapoint *myPoint = malloc(sizeof(Megapoint);
myPoint->w = 42.0f;
// set ivars as desired..

// Casting to "id" to avoid compiler warning
[array addObject:(id)myPoint];

// Hurray!
Megapoint *mySamePoint = [array objectAtIndex:0];

Solution 3

A similar method to add c struct is to store the pointer and to de-reference the pointer as so;

typedef struct BSTNode
{
    int data;
    struct BSTNode *leftNode;
    struct BSTNode *rightNode;
}BSTNode;

BSTNode *rootNode;

//declaring a NSMutableArray
@property(nonatomic)NSMutableArray *queues;

//storing the pointer in the array
[self.queues addObject:[NSValue value:&rootNode withObjCType:@encode(BSTNode*)]];

//getting the value
BSTNode *frontNode =[[self.queues objectAtIndex:0] pointerValue];

Solution 4

if you're feeling nerdy, or really have a lot of classes to create: it is occasionally useful to dynamically construct an objc class (ref: class_addIvar). this way, you can create arbitrary objc classes from arbitrary types. you can specify field by field, or just pass the info of the struct (but that's practically replicating NSData). sometimes useful, but probably more of a 'fun fact' for most readers.

How would I apply this here?

you can call class_addIvar and add a Megapoint instance variable to a new class, or you can synthesize an objc variant of the Megapoint class at runtime (e.g., an instance variable for each field of Megapoint).

the former is equivalent to the compiled objc class:

@interface MONMegapoint { Megapoint megapoint; } @end

the latter is equivalent to the compiled objc class:

@interface MONMegapoint { float w,x,y,z; } @end

after you've added the ivars, you can add/synthesize methods.

to read the stored values on the receiving end, use your synthesized methods, object_getInstanceVariable, or valueForKey:(which will often convert these scalar instance variables into NSNumber or NSValue representations).

btw: all the answers you have received are useful, some are better/worse/invalid depending on the context/scenario. specific needs regarding memory, speed, ease to maintain, ease to transfer or archive, etc. will determine which is best for a given case... but there is no 'perfect' solution which is ideal in every regard. there is no 'best way to put a c-struct in an NSArray', just a 'best way to put a c-struct in an NSArray for a specific scenario, case, or set of requirements' -- which you'd have to specify.

furthermore, NSArray is a generally reusable array interface for pointer sized (or smaller) types, but there are other containers which are better suited for c-structs for many reasons (std::vector being an typical choice for c-structs).

Solution 5

it would be best to use the poor-man's objc serializer if you're sharing this data across multiple abis/architectures:

Megapoint mpt = /* ... */;
NSMutableDictionary * d = [NSMutableDictionary new];
assert(d);

/* optional, for your runtime/deserialization sanity-checks */
[d setValue:@"Megapoint" forKey:@"Type-Identifier"];

[d setValue:[NSNumber numberWithFloat:mpt.w] forKey:@"w"];
[d setValue:[NSNumber numberWithFloat:mpt.x] forKey:@"x"];
[d setValue:[NSNumber numberWithFloat:mpt.y] forKey:@"y"];
[d setValue:[NSNumber numberWithFloat:mpt.z] forKey:@"z"];

NSArray *a = [NSArray arrayWithObject:d];
[d release], d = 0;
/* ... */

...particularly if the structure can change over time (or by targeted platform). it's not as fast as other options, but it's less likely to break in some conditions (which you haven't specified as important or not).

if the serialized representation does not exit the process, then size/order/alignment of arbitrary structs should not change, and there are options which are simpler and faster.

in either event, you're already adding a ref-counted object (compared to NSData, NSValue) so... creating an objc class which holds Megapoint is the right answer in many cases.

View more solutions

Comments

[Catfish_Man]

It's just storing the pointer; doesn't touch the object (including retaining or copying it) at all. I suggest + (NSValue *)valueWithPointer:(const void *)aPointer instead of +valueWitHBytes:objCType: though.

[Justin Spahr-Summers]

@Joe Blow @Catfish_Man It actually copies the structure p, not a pointer to it. The @encode directive provides all the information necessary about how big the structure is. When you release the NSValue (or when the array does), its copy of the structure is destroyed. If you've used getValue: in the meantime, you're fine. See the "Using Values" section of "Number and Value Programming Topics": developer.apple.com/library/ios/documentation/Cocoa/Conceptu‌​al/…

[Justin Spahr-Summers]

@Joe Blow No, valueWithPointer: is used specifically to encapsulate a pointer value in an object. It knows nothing about what type of object it's pointing to (if any). valueWithBytes:objCType: is used to encapsulate any known-size value in an object. One illustrative example is that valueWithPointer: would work with NULL, while valueWithBytes:objCType: would not.

[Justin Spahr-Summers]

@Joe Blow The data stored in an NSValue object will never change, and its size can be determined at compile-time, both of which may lend themselves to certain optimizations.

[Justin Spahr-Summers]

@Joe Blow Mostly correct, except that it wouldn't be able to change at runtime. You're specifying a C type, which always must be fully known. If it could get "larger" by referencing more data, then you would probably implement that with a pointer, and the @encode would describe the structure with that pointer, but not fully describe the pointed-to data, which could indeed change.

[justin]

@Joe Blow something which performs serialization. for reference: en.wikipedia.org/wiki/Serialization, parashift.com/c++-faq-lite/serialization.html, as well as Apple's "Archives and Serializations Programming Guide".

[Justin Spahr-Summers]

@Joe Blow Yep, that last one pretty much sums it up. NSData has its mutable subclass too, which is even more flexible but less efficient unless you're doing lots of appending or inserting.

[Evgen Bodunov]

STL is a library partially included into C++ Standard Library. en.wikipedia.org/wiki/Standard_Template_Library cplusplus.com/reference/stl/vector

[justin]

people's backgrounds come into play as well... how you need to use that struct will often eliminate some possibilities. 4 floats is pretty foolproof, but struct layouts vary by architecture/compiler too much to use a contiguous memory representation (e.g., NSData) and expect it to work. the poor man's objc serializer likely has the slowest execution time, but it is the most compatible if you need to save/open/transmit the Megapoint on any OS X or iOS device. The most common way, in my experience is to simply put the struct in an objc class. if you're going through all of this just to (cont)

[justin]

(cont) If you're going through all this hassle only to avoid learning a new collection type -- then you should learn the new collection type =) std::vector (for example) is more suited to holding C/C++ types, structs and classes than NSArray. By using an NSArray of NSValue, NSData, or NSDictionary types, you're losing a lot of type-safety while adding a ton of allocations and runtime overhead. If you want to stick with C, then they'd generally use malloc and/or arrays on the stack... but std::vector hides most of the complications from you.

[Justin Spahr-Summers]

@Joe Blow The STL requires that you use C++ or Objective-C++ for your project. If you're already doing so or planning to do so, using std::vector is no problem. But C++ certainly has its own problems, and integrating it with Objective-C can get rather ugly. I would recommend against using Objective-C++ unless you have a third-party library (for instance) that is written in C++.

[justin]

in fact, if you want array manipulation/iteration as you mentioned - stl (part of c++ standard libraries) is great for that. you have more types to choose from (e.g., if insert/remove is more important than read access times), and tons of existing ways to manipulate the containers. also - it's not bare memory in c++ - the containers and template functions are type-aware, and checked at compilation - much safer than pulling an arbitrary byte string out of NSData/NSValue representations. they also have bounds checking and mostly automatic management of memory. (cont)

[justin]

(cont) if you expect that you'll have a lot of low level work like this, then you should just learn it now - but it will take time to learn. by wrapping this all up in objc representations, you're losing a lot of performance and type safety, while making yourself write much more boilerplate code to access and interpret the containers and their values (If 'Thus, to be very specific…' is exactly what you want to do).

[Sedate Alien]

That's an interesting claim. Do you have a link to an article about the speed advantage of the STL containers vs. the Cocoa container classes?

[Sedate Alien]

Mutable arrays (at least in this case) are created in an empty state and hence don't need a pointer to the values to be stored within. This is distinct from the immutable array wherein the contents are "frozen" at creation and hence values must be passed to the initialisation routine.

[Sedate Alien]

@Joe Blow: That's an excellent point you make regarding memory management. You are right to be confused: the code sample I posted above would cause mysterious crashes, depending on when the function's stack is overwritten. I started to elaborate on how my solution might be used, but realised that I was reimplementing Objective-C's own reference-counting. My apologies for the compact code -- it's not a matter of aptitude but laziness. No point in writing code that others can't read. :)

[Sedate Alien]

If you were happy to "leak" (for want of a better word) the struct, you could certainly allocate it once and not free it in the future. I've included an example of this in my edited answer. Also, it was not a typo for myStruct, as it was a structure allocated on the stack, as distinct from a pointer to a structure allocated on the heap.

[justin]

it's just another tool at your disposal. there can be complications integrating objc with c++, c++ with objc, c with c++ or any of several other combinations. adding language features and using multiple languages does come at a small cost in any case. it goes all ways. for example, build times go up when compiling as objc++. as well, these sources are not reused in other projects as easily. sure, you can reimplement language features... but that's not often the best solution. integrating c++ in an objc project is fine, it's about as 'messy' as using objc and c sources in the same project.(cont

[justin]

the objc interfaces must be guarded, so they are not visible to c translations (compilations) - same concept, but you're adding more dialects/languages. if your codebase is mostly objc/c++ - then you may prefer to use objc++ all the time. if you only use it in some places, then it may be easiest to hide the c++ pieces behind a c or objc interface. i use objc++ often - there aren't faults with the objc++ implementation/runtime - integration with other sources is the primary concern. just use the tools that work best for your needs and don't be afraid to integrate other languages, if needed.

[justin]

assuming the xml file properly represents something, then yes - it's one common form of a human-readable serialization.

[justin]

in my codebase, c++ is the most common language so... i end up only having to hide objc interfaces from c++ translations in most cases. in the event a class needs to be used in c or obc (no ++), then i wrap what i need in a c or objc interface. this cuts down most of the noise.

[justin]

the primary problem with tidy handling of arrays in C are type safety and use of the preprocessor. so you're either passing through void* and severely limiting your options, or using a ton of preprocessor expansion which cripples many tools' abilities to evaluate your program. in short, 'no, i can't recommend one', c++ has already solved these problems, so i use that. one alternative i do use from time to time is to write a simple code generator, when i really need to stick with C or ObjC. if you're really curious, it's worth a new question here @SO. somebody knows.

[justin]

here's an interesting read on NSCFArray vs std::vector: ridiculousfish.com/blog/archives/2005/12/23/array in the example in your post, the biggest loss is (typically) creating an objc object representation per element (e.g., NSValue, NSData, or Objc type containing Megapoint requires an allocation and insertion into ref-counted system). you could actually avoid that by using Sedate Alien's approach to storing a Megapoint in a special CFArray which uses a separate backing store of contiguously allocated Megapoints (although neither example illustrates that approach). (cont)

[justin]

but then using the NSCFArray vs the vector (or other stl type) will incur additional overhead for dynamic dispatch, additional function calls which aren't inlined, a ton of type safety, and many chances for the optimizer to kick in... that article just focuses on insert, read, walk, delete. chances are, you won't get any faster than a 16 byte aligned c-array Megapoint pt[8]; - this is an option in c++, and specialized c++ containers (e.g., std::array) - also note the example does not add the special alignment (16 bytes was chosen because it is the size of Megapoint). (cont)

[justin]

std::vector will add a tiny amount of overhead to this, and one allocation (if you know the size you'll need)... but this is closer to the metal than more than 99.9% of cases need. typically, you'd just use a vector unless the size is fixed or has a reasonable maximum.

[Evgen Bodunov]

I agreed. After testing your data use on your specific data amount and size you will see what container is fits better.

[Justin Spahr-Summers]

@Joe Blow I would create (a) separate question(s) if you want to investigate the topics of Objective-C++ and std::vector alternatives further. Then others, who might have more insight than I, would be able to jump in too.

[devios1]

Will NSValue automatically free the memory of the struct when it is deallocated? The documentation is a little unclear on this.

[Justin Spahr-Summers]

@devios The contents of the structure are copied into the NSValue. The pointer you pass in is not retained.

[devios1]

So just to be completely clear, the NSValue owns the data it copies into itself and I don't have to worry about freeing it (under ARC)?

[Justin Spahr-Summers]

@devios Correct. NSValue doesn't really do any “memory management,” per se—you can think of it as just having a copy of the structure value internally. If the structure contained nested pointers, for example, NSValue wouldn't know to free or copy or do anything with those—it would leave them untouched, copying the address as-is.

[Ameet Dhas]

how do I get this value back in Swift 3?

[Fattie]

this gets the ten year old question badge!