Map with concurrent access

Question

When you use a map in a program with concurrent access, is there any need to use a mutex in functions to read values?

Solution 1

Multiple readers, no writers is okay:

https://groups.google.com/d/msg/golang-nuts/HpLWnGTp-n8/hyUYmnWJqiQJ

One writer, no readers is okay. (Maps wouldn't be much good otherwise.)

Otherwise, if there is at least one writer and at least one more either writer or reader, then all readers and writers must use synchronization to access the map. A mutex works fine for this.

Solution 2

sync.Map has merged to Go master as of April 27, 2017.

This is the concurrent Map we have all been waiting for.

https://github.com/golang/go/blob/master/src/sync/map.go

https://godoc.org/sync#Map

Solution 3

I answered your question in this reddit thread few days ago:

In Go, maps are not thread-safe. Also, data requires locking even for reading if, for example, there could be another goroutine that is writing the same data (concurrently, that is).

Judging by your clarification in the comments, that there are going to be setter functions too, the answer to your question is yes, you will have to protect your reads with a mutex; you can use a RWMutex. For an example you can look at the source of the implementation of a table data structure (uses a map behind the scenes) which I wrote (actually the one linked in the reddit thread).

Solution 4

You could use concurrent-map to handle the concurrency pains for you.

// Create a new map.
map := cmap.NewConcurrentMap()

// Add item to map, adds "bar" under key "foo"
map.Add("foo", "bar")

// Retrieve item from map.
tmp, ok := map.Get("foo")

// Checks if item exists
if ok == true {
    // Map stores items as interface{}, hence we'll have to cast.
    bar := tmp.(string)
}

// Removes item under key "foo"
map.Remove("foo")

Solution 5

if you only have one writer, then you can probably get away with using an atomic Value. The following is adapted from https://golang.org/pkg/sync/atomic/#example_Value_readMostly (the original uses locks to protect writing, so supports multiple writers)

type Map map[string]string
    var m Value
    m.Store(make(Map))

read := func(key string) (val string) { // read from multiple go routines
            m1 := m.Load().(Map)
            return m1[key]
    }

insert := func(key, val string) {  // update from one go routine
            m1 := m.Load().(Map) // load current value of the data structure
            m2 := make(Map)      // create a new map
            for k, v := range m1 {
                    m2[k] = v // copy all data from the current object to the new one
            }
            m2[key] = val // do the update that we need (can delete/add/change)
            m.Store(m2)   // atomically replace the current object with the new one
            // At this point all new readers start working with the new version.
            // The old version will be garbage collected once the existing readers
            // (if any) are done with it.
    }

View more solutions

Comments

[jimt]

If it is strictly a read-only map, a mutex should not be necessary.

[Admin]

I was not very clear since there will be functions to set and get values.

[Lynn]

it's usually wasteful to use a full reader-writer lock for a resource as fast to access as a map

[Admin]

Can you elaborate a bit please? What would be better suited?

[Lynn]

RW locks are good for resources with a lot of contention, but they have more overhead than a mutex. Map get/sets are fast enough that the program likely won't have enough contention to make the more complex lock give better throughput than a simple mutex.

[Admin]

Thank you for the clarification. Do you have any papers/articles that you can recommend on this matter?

[fiorix]

However, the Concurrency section of this post blog.golang.org/go-maps-in-action suggests sync.RWMutex

[Lucretiel]

Things like this are why I can't take seriously the notion that go "doesn't need generics."

[Alexander]

The notion isn't that Go "doesn't need generics" but "there is currently no clean way to implement generics, we need to think about this some more". C++ for example generates code for all possible combination of types that are being used, which increases compilation times and executable sizes by an unreasonable amount.

[orcaman]

Nice one. Do note that the new sync.Map type is designed for append-only maps (for this reason it doesn't use sharding of keys, which means that if your map has a lot of churn it will most likely underperform sharded style maps as per my answer above).

[orcaman]

@OmarIlias I mean cases where you mainly set new values (without editing or deleting existing values). See this: github.com/golang/go/issues/20360

[Diegomontoya]

@orcaman current go sync.Map can be faster or slower than hash-sharded even in your append-only case. It really depends on the scenario. Using as much atomic operations as possible, the sync.Map internals can be much faster than traditional sharding since it minimize locking versus traditional hash-bucket but required locking. I am sure there will be benchmarks performed which detail the sweet and sore spots of the new sync.Map implementation. But to assume is slower is not correct especially factoring concurrency.

[user3125693]

What about multiple writers, no readers? Would that result in memory corruption?

[John Balvin Arias]

@Diegomontoya of course, here it is: medium.com/@deckarep/…, in brief sync.map will be faster only if the number of cores used is bigger than 4, if not the case, using only mutext will be 4 times faster at maximum

[John Balvin Arias]

Could my app crashed if I don´t use any synchronization with multiple readers and one writer? or it´s just that the map could not get updated?

[John Balvin Arias]

never mind, I find my answer, and yes it will crashed golang.org/doc/go1.6#runtime

[hqt]

@user3125693 Sure.