Why does Go forbid taking the address of (&) map member, yet allows (&) slice element?

Question

Go doesn't allow taking the address of a map member:

// if I do this:
p := &mm["abc"]
// Syntax Error - cannot take the address of mm["abc"]

The rationale is that if Go allows taking this address, when the map backstore grows or shinks, the address can become invalid, confusing the user.

But Go slice gets relocated when it outgrows its capacity, yet, Go allows us to take the address of a slice element:

 a := make([]Test, 5)
 a[0] = Test{1, "dsfds"}
 a[1] = Test{2, "sdfd"}
 a[2] = Test{3, "dsf"}

 addr1 := reflect.ValueOf(&a[2]).Pointer()
 fmt.Println("Address of a[2]: ", addr1)

 a = append(a, Test{4, "ssdf"})
 addrx := reflect.ValueOf(&a[2]).Pointer()
 fmt.Println("Address of a[2] After Append:", addrx)

 // Note after append, the first address is invalid
 Address of a[2]:  833358258224
 Address of a[2] After Append: 833358266416

Why is Go designed like this? What is special about taking address of slice element?

Solution 1

There is a major difference between slices and maps: Slices are backed by a backing array and maps are not.

If a map grows or shrinks a potential pointer to a map element may become a dangling pointer pointing into nowhere (uninitialised memory). The problem here is not "confusion of the user" but that it would break a major design element of Go: No dangling pointers.

If a slice runs out of capacity a new, larger backing array is created and the old backing array is copied into the new; and the old backing array remains existing. Thus any pointers obtained from the "ungrown" slice pointing into the old backing array are still valid pointers to valid memory.

If you have a slice still pointing to the old backing array (e.g. because you made a copy of the slice before growing the slice beyond its capacity) you still access the old backing array. This has less to do with pointers of slice elements, but slices being views into arrays and the arrays being copied during slice growth.

Note that there is no "reducing the backing array of a slice" during slice shrinkage.

Solution 2

A fundamental difference between map and slice is that a map is a dynamic data structure that moves the values that it contains as it grows. The specific implementation of Go map may even grow incrementally, a little bit during insert and delete operations until all values are moved to a bigger memory structure. So you may delete a value and suddenly another value may move. A slice on the other hand is just an interface/pointer to a subarray. A slice never grows. The append function may copy a slice into another slice with more capacity, but it leaves the old slice intact and is also a function instead of just an indexing operator.

In the words of the map implementor himself:

https://www.youtube.com/watch?v=Tl7mi9QmLns&feature=youtu.be&t=21m45s "It interferes with this growing procedure, so if I take the address of some entry in the bucket, and then I keep that entry around for a long time and in the meantime the map grows, then all of a sudden that pointer points to an old bucket and not a new bucket and that pointer is now invalid, so it's hard to provide the ability to take the address of a value in a map, without constraining how grow works... C++ grows in a different way, so you can take the address of a bucket"

So, even though &m[x] could have been allowed and would be useful for short-lived operations (do a modification to the value and then not use that pointer again), and in fact the map internally does that, I think the language designers/implementors chose to be on the safe side with map, not allowing &m[x] in order to avoid subtle bugs with programs that might keep the pointer for a long time without realizing then it would point to different data than the programmer thought.

See also Why doesn't Go allow taking the address of map value? for related comments.

Comments

[NeoWang]

Thanks! How about struct fields? I can take the address of a field like this: ptr := &(object.Field1), but when I try to get the address from reflect, CanAddr() return false: I tried reflect.ValueOf(object).FieldByName("Field1").CanAddr() and reflect.ValueOf(&object).Elem().FieldByName("Field1").CanAdd‌​r(), both returns false. Why?

[JimB]

@NeoWang: because you're passing the value of object to reflect. You can ask another separate question if you still don't understand; it's not related to maps or slices.

[mugi]

I don't think it is the correct answer. The address of slice elements may also change. But GO allows to take the address of slice elements. There is no dangling pointer in GO. All pointers are managed.

[ransomenote]

If you think about the address of the element as the address of the slot where this element is stored, maps are no longer that different from slices. In both cases you can observe different value after mutation, with map being different in that it's also possible to see zero value (like if the key would be missing). Arguably both behaviors lead to similar bugs, which are still less severe than data race bugs in general where interleaved read/writes start producing new values. As @mugi mentioned no dangling pointers are possible in absence of data races.

[Volker]

@pingw33n Maps and slices in Go are fundamental different according to the language specification. Also terms like "address" has a very well defined and concrete meaning in Go. There are no dangling pointers in Go even in the presence of data races.

[Volker]

@mugi The address of a slice element does not "change" as the element is addressable. If you construct a new slice (e.g. via append) its elements will have different addresses but the address of the elements of the old slice do not change.

[ransomenote]

Go is memory safe only when there's no data races blog.stalkr.net/2015/04/…. This is something you can't avoid within current language semantics and implementation. /// Go specs defines "addressable" but doesn't add to "address" any special meaning. Address is just an offset in linear memory space and not a reference to specific identifiable value (despite in most cases it's the effective semantics). Aliasing allows simultaneous memory reads & writes producing effects I described above. Non-addressable map seems an artificial restriction.

[Valer]

the address of slice[x] will change on reallocation of the underlying array. ofc, references to the old elements still remain intact. play.golang.org/p/pX6_wp0bvVE