Implementing Reader interface

Question

I understand the general concept of Go's interfaces. However, I was recently looking into implementing the io.Reader interface, and it has me confused. I found this post which didn't help to much.

Reader interface and the Read method in golang

To start with, the accepted answer is using io.Reader's Read function, which as far as I can tell is never implemented. Second, how does the Read function work in the context of something like ioutil.ReadAll. It takes something that implemented the io.Reader interface and returns a slice of bytes. I don't understand how something that is returning only an int and err can be processed into a slice of bytes.

Edit:

I was helped in the go-nuts IRC channel and this was the provided answer as to how you would likely actually implement one http://play.golang.org/p/ejpUVOx8jR. Much thanks to the go community.

Edit 2:

as pointed out below the implementation above will fail in the case where the strign is larger than the buffer. This is a more sane implementation http://play.golang.org/p/t4Zg8TnF33.

Solution 1

You pass Read the byte slice. Read is supposed to put bytes in it. As slices are just references to arrays, changing the contents of a slice changes the underlying array, so the caller of Read can then just check the slice it has passed to it.

ioutil.ReadAll creates a buffer and calls ReadFrom on it. ReadFrom calls Read repeatedly, increasing the size of the buffer until Read tells it has been exhausted by returning io.EOF as error. See for yourself.

The answer you link does implement the io.Reader interface. It is declaring a method Read(p []byte) (n int, e error). That's all what is needed.

Solution 2

The updated answer provided by tez totally works, but here is an alternative that I think is a bit cleaner utilizing Go's copy:

type Reader struct {
    data []byte
    readIndex int64
}

func (r *Reader) Read(p []byte) (n int, err error) {
    if r.readIndex >= int64(len(r.data)) {
        err = io.EOF
        return
    }

    n = copy(p, r.data[r.readIndex:])
    r.readIndex += int64(n)
    return
}

By using copy, you don't have to worry about overflowing p []byte. This also doesn't drain/destroy any state that you have on your reader, instead it just iterates over it with readIndex.

Full example here: https://play.golang.org/p/8QTECCkies

This strategy can be seen in some of Go's core packages (ie. https://golang.org/src/strings/reader.go)

Solution 3

Just to note, your implementation of Read() method here (http://play.golang.org/p/ejpUVOx8jR) is incorrect. You do not account for a capacity of the provided p []byte argument. You'll get an "index out of range" panic if it is smaller than your test string (ex: http://play.golang.org/p/DhcY0hJ0c0).

You might consider a different approach such as this - http://play.golang.org/p/t4Zg8TnF33.

UPDATE I noticed a memory leak in my implementation. The fixed version is here - http://play.golang.org/p/9BbS54d8pb. This goes to show that even a trivial code sometimes is not so trivial :)

Comments

[eatingthenight]

Awesome thank you. I will update my answer so others don't implement this poorly.