Go Patterns and Practices

2022-09-19

Hi

I’m Abhinav.

I work on Go frameworks.

Let’s talk about writing Go.

Agenda

  • Rules

  • Why

  • Patterns

  • Practices

Rules

  1. Exported means forever

  2. Function types are fixed

  3. Interfaces are immutable

Exported means forever

Cannot remove

functions
variable
type
constants

once exported

Function types are fixed

Cannot modify

function
or
method

signatures

 func DeleteUser(
   ctx context.Context,
   name string,
+  softDelete bool,
 ) (...)

adding a parameter

 func DeleteUser(...) (
  deleted bool,
+ uuid uuid.UUID,
  err  error,
 )

or a return value

// ERROR
// - missing parameter: softDelete
// - cannot assign 3 values to 2 variables
deleted, err := DeleteUser(ctx, name)
if err != nil {
  // ...
}

breaks existing callers

Variadic arguments

 func ListPosts(
   ctx context.Context,
+  skipAuthors ...string
 ) ([]*Post, error)

This is fine, right?

posts, err := ListPosts(ctx)
if err != nil {
  // ...
}

Nope!

func Process(
  ctx context.Context,
  listPosts func(...) ([]*Post, error)
) {
  posts, err := listPosts(ctx)
  // ...
}

Breaks function references

Process(ctx, ListPosts)
// ERROR:
// want func(Context) (...)
//  got func(Context, ...string) (...)

Interfaces are immutable

Cannot

modify
remove
add

methods

Adding methods to an interface

 type Writer interface {
   WriteBytes([]byte) error
+  WriteString(string) error
 }
func WriteTo(w Writer) error {
  // ...
}

breaks existing implementations

type myWriter struct{ /* ... */ }

func (w *myWriter) WriteBytes(
  b []byte,
) error {
  // ...
}
// ERROR:
// does not implement WriteString
err := WriteTo(&myWriter{...})
if err != nil {
  // ...
}

Rules

  1. Exported means forever

  2. Function types are fixed

  3. Interfaces are immutable

Patterns

Exported means forever

  • Be deliberate

  • You can always export it later

Function types are fixed

How to plan for expansion?

Parameter Objects

 type DeleteUserRequest struct {
   Name       string
+  SoftDelete bool
 }
func DeleteUser(
  ctx context.Context,
  req DeleteUserRequest,
) (...) {
  // ...
}

struct exclusively for parameters except Context

New optional fields

Need more than 3 parameters?
Use a struct.

Result Objects

 type DeleteUserResponse struct {
   Deleted bool
+  UUID    uuid.UUID
 }
func DeleteUser(
  ...
) (DeleteUserResponse, error) {
  // ...
}

struct exclusively for returns except error

New fields for new outputs

Need more than 2 results?
Use a struct.

Functional Options

type Option /* ... */

func SkipAuthors(
  names ...string,
) Option { /* ... */ }

func ListPosts(
  ctx context.Context,
  opts ...Option,
) ([]*Post, error)
ListPosts(ctx)

ListPosts(ctx,
  SkipAuthors(...),
)
type Option /* ... */

func SkipAuthors(
  names ...string,
) Option { /* ... */ }

func Archived(
  bool,
) Option { /* ... */ }

func ListPosts(
  ctx context.Context,
  opts ...Option,
) ([]*Post, error)
ListPosts(ctx)

ListPosts(ctx,
  Archived(true),
  SkipAuthors(...),
)

ListPosts(ctx,
  Archived(false),
)

Do

  • Zero argument default

  • Required positional arguments

Don’t

  • Required options

  • Mix with parameter objects

How to implement functional options

Setup

type options struct {
}

type Option interface {
  apply(*options)
}

Add an option

type options struct {
  skipAuthors []string
}

type Option interface {
  apply(*options)
}
func SkipAuthors(
  xs ...string,
) Option {
  return skipAuthors(xs)
}

type skipAuthors []string

func (o skipAuthors) apply(...) {
  opts.skipAuthors = []string(o)
}

Add other options

type options struct {
  skipAuthors []string
  archived    bool
  postedAfter time.Time
}

type Option interface {
  apply(*options)
}
func SkipAuthors(
  xs ...string,
) Option { /* ... */ }

func Archived(
  archived bool,
) Option { /* ... */ }

func PostedAfter(
  postedAfter date.Date,
) Option { /* ... */ }

Consume the options

type options struct {
  skipAuthors []string
  archived    bool
  postedAfter time.Time
}

type Option interface {
  apply(*options)
}
func ListPosts(
  ctx context.Context,
  os ...Option,
) (...) {
  var opts options
  for _, o := range os {
    o.apply(&opts)
  }
  if opts.archived {
    // ...
  }

Return new results

type options struct {
  skipAuthors []string
  archived    bool
  postedAfter time.Time
  queryStats *Stats
}

type Stats struct {
  Elapsed time.Duration
  /* ... */
}

func QueryStats(*Stats) Option {
  /* ... */
}
var stats Stats
posts, err := ListPosts(ctx,
  QueryStats(&stats),
)
if err != nil {
  // ...
}

log.Printf(
  "Query took %v", stats.Elapsed)
// Output:
// Query took 100ms

Summary

  • Add new parameters or results

  • Compose options together

    func(...Option) Option

High flexibility and high boilerplate

Plan for function expansion

Parameter Objects
Result Objects
Functional Options
New Functions

Interfaces are immutable

How to plan for expansion?

Keep interfaces small

Bad

type Writer interface {
  WriteBytes([]byte) error
  WriteString(string) error
}

func (w *myWriter) WriteString(
  s string,
) error {
  return w.WriteBytes([]byte(s))
}

Good

type Writer interface {
  WriteBytes([]byte) error
}

func WriteString(
  w Writer, s string,
) error {
  return w.WriteBytes([]byte(s))
}

No helper methods — use functions

Upcast to upgrade

type Writer interface {
  WriteBytes([]byte) error
}

func WriteString(w Writer, s string) error {
  return w.WriteBytes([]byte(s))
}

Add a new interface

type Writer interface {
  WriteBytes([]byte) error
}

type StringWriter interface {
  WriteString(string) error
}

func WriteString(w Writer, s string) error {
  return w.WriteBytes([]byte(s))
}

Upcast and use implementation

type Writer interface {
  WriteBytes([]byte) error
}

type StringWriter interface {
  WriteString(string) error
}

func WriteString(w Writer, s string) error {
  if sw, ok := w.(StringWriter); ok {
    return sw.WriteString(s)
  }
  return w.WriteBytes([]byte(s))
}

io/fs

type FS interface{
  Open(string) (File, error)
}
func Stat(fs FS, name string) (FileInfo, error) {
  f, err := fs.Open(name)
  if err != nil {
    return nil, err
  }
  return f.Stat()
}

io/fs

type FS interface{
  Open(string) (File, error)
}
type StatFS interface {
  FS

  Stat(string) (FileInfo, error)
}
func Stat(fs FS, name string) (FileInfo, error) {
  if sf, ok := fs.(StatFS); ok {
    return sf.Stat(name)
  }

  f, err := fs.Open(name)
  if err != nil {
    return nil, err
  }
  return f.Stat()
}

io/fs

type FS interface{
  Open(string) (File, error)
}
type ReadFileFS interface {
  FS

  ReadFile(string) ([]byte, error)
}
func ReadFile(fs FS, name string) ([]byte, error) {
  if rf, ok := fs.(ReadFileFS); ok {
    return rf.ReadFile(name)
  }

  f, err := fs.Open(name)
  if err != nil {
    return nil, err
  }
  return io.ReadAll(f)
}

Summary

Pros
  • DRY implementation

  • Upgrade as needed

Cons
  • No internal state

  • Wrapping breaks overrides

Driver Interface

Diagram
Diagram
Diagram
type Driver interface {
  WriteBytes([]byte) error
}
type Writer struct{ drv Driver }

func (w *Writer) WriteBytes(bs []byte) error {
  return w.drv.WriteBytes(bs)
}
type Writer struct{ drv Driver }

func (w *Writer) WriteBytes(bs []byte) error {
  return w.drv.WriteBytes(bs)
}

func (w *Writer) WriteString(bs []byte) error {
  return w.WriteBytes([]string(bs))
}
type StringWriter interface {
  WriteString(string) error
}
type Writer struct{ drv Driver }

func (w *Writer) WriteBytes(bs []byte) error {
  return w.drv.WriteBytes(bs)
}

func (w *Writer) WriteString(bs []byte) error {
  if sw, ok := w.drv.(StringWriter); ok {
    return sw.WriteString(s)
  }
  return w.WriteBytes([]string(bs))
}

Benefits

  • Implement a small interface

  • Consume a large surface

  • Store internal state

Examples

  • zap.Logger wraps zapcore.Core

  • http.Client wraps http.RoundTripper

  • database/sql wraps database/sql/driver

Practices

Goroutines

Lightweight concurrency
but not free

Each goroutine costs at least 2 KB [1]
1. runtime/stack.go L72, L1479

Lifetime control

All goroutines

must finish or
must be stoppable

Do not fire-and-forget

Bad

go func() {
  for {
    flush()
    time.Sleep(delay)
  }
}()
Cannot be stopped

Good

stop := make(chan struct{})
go func() {
  tick := time.NewTicker(delay)
  defer tick.Stop()
  for {
    select {
    case <-tick.C:
      flush()
    case <-stop:
      return
    }
  }
}()

Stop with close(stop)

Bad

go func() {
  for {
    flush()
    time.Sleep(delay)
  }
}()
Cannot be stopped

Better

ctx, cancel := /* ... */
go func() {
  tick := time.NewTicker(delay)
  defer tick.Stop()
  for {
    select {
    case <-tick.C:
      flush(ctx)
    case <-ctx.Done():
      return
    }
  }
}()

Stop with cancel()

What if I need fire-and-forget?

Use a worker pool

func worker(
  jobc <-chan job,
) {
  for job := range jobc {
    job.do()
  }
}
Runs until close(jobc)
type manager struct {
  jobc chan<- job
}
mgr := manager{
  jobc: jobc,
}
for i := 0; i < NumWorkers; i++ {
  go worker(jobc)
}
func (m *manager) Stop() {
  close(m.jobc)
}
func worker(
  ctx context.Context,
  jobc <-chan job,
) {
  for {
    select {
    case <-ctx.Done():
      return
    case job := <-jobc:
      job.do(ctx)
    }
  }
}
Runs until close(jobc).
type manager struct {
  stop context.CancelFunc
  jobc chan<- job
}
ctx, cancel := context.WithCancel(..)
mgr := manager{
  stop: cancel,
  jobc: jobc,
}
for i := 0; i < NumWorkers; i++ {
  go worker(ctx, jobc)
}
func (m *manager) Stop() {
  m.stop()
}
func worker(
  ctx context.Context,
  jobc <-chan job,
  wg   *sync.WaitGroup,
) {
  defer wg.Done()
  for {
    select {
    case <-ctx.Done():
      return
    case job := <-jobc:
      job.do(ctx)
    }
  }
}
type manager struct {
  stop context.CancelFunc
  jobc chan<- job
  wg   sync.WaitGroup
}
ctx, cancel := context.WithCancel(..)
mgr := manager{
  stop: cancel,
  jobc: jobc,
}
mgr.wg.Add(NumWorkers)
for i := 0; i < NumWorkers; i++ {
  go worker(ctx, jobc, &mgr.wg)
}
func (m *manager) Stop() {
  m.stop()
  m.wg.Wait()
}

Errors

Produce errors

rootcause

Leaf errors

Not a leaf error

out, err := f()
if err != nil {
  return err
}

out, err := g()
if err != nil {
  return fmt.Errorf("g: %v", err)
}

Leaf errors

return errors.New("great sadness")

return fmt.Errorf(
  "unhappy result %q", result)

Can they be handled?

No

Use fmt.Errorf / errors.New
return errors.New("session is closed")

return fmt.Errorf(
  "invalid email address %q", email)

Yes

  • Sentinel errors

  • Structured errors

Sentinel errors

Global error variables

package fs

var ErrNotExist =
  errors.New("file does not exist")

var ErrExist =
  errors.New("file already exists")

var ErrPermission =
  errors.New("permission denied")
  • Prefix name with "Err"

  • Avoid overly verbose names (ErrFileAlreadyExists)

  • Match with errors.Is

    if errors.Is(err, fs.ErrNotExist) {
      createFile(..)
    }

Structured errors

Error types

type SyntaxError struct {
  Line, Column int
  msg          string
}

func (e *SyntaxError) Error() string
  • Suffix with "Error"

  • Use a pointer receiver

  • Match with errors.As

    var synErr *SyntaxError
    if errors.As(err, &synErr) {
      highlight(synErr.Line)
    }

Consume errors

Propagate:

return to caller

Handle:

react to it

Propagate errors

out, err := f()
if err != nil {
  return err
}
out, err := g()
if err != nil {
  return fmt.Errorf("get g: %w")
}
Use %w to retain error matching
type FooError struct {
  ID  string
  Err error
}

func (e *FooError) Unwrap() error {
  return e.Err
}

out, err := h(id)
if err != nil {
  return &FooError{ID: id, Err: err}
}
Add Unwrap() to retain error matching

Error messages

No "failed to X"

fmt.Errorf("failed to load config: %w", err) // BAD
// failed to load config: failed to decode protobuf: ...

fmt.Errorf("load config: %w", err)           // GOOD
// load config: decode protobuf: ...

Use %q

fmt.Errorf("get user %v: %w", user, err) // BAD
// get user : invalid name

fmt.Errorf("get user %q: %w", user, err) // GOOD
// get user "": invalid name

Don’t capitalize

fmt.Errorf("Send request: %w", err) // BAD
// Send request: Connect to "foo": ...

fmt.Errorf("send request: %w", err) // GOOD
// send request: connect to "foo": ...

Handle errors

sentinel errors ⇒ errors.Is

func loadConfig(..) (*Config, error) {
  f, err := os.Open(name)
  if err != nil {
    if errors.Is(err, fs.ErrNotExist) {
      // Use default configuration
      // if file does not exist.
      return _defaultConfig, nil
    }

    // Fail on all other errors.
    return nil, err
  }
  defer f.Close()

  // ...
}

structured errors ⇒ errors.As

c, err := connect()
if err != nil {
  var dnsErr *net.DNSError
  if errors.As(err, &dnsErr) && len(conns) {
    // Fallback to a random existing
    // connection, if any, if DNS resolution
    // failed.
    log.Warn(..., dnsErr.Name)
    c = conns[rand.Intn(len(conns))]
  } else {
    // Fail on all other errors.
    return nil, err
  }
}

Summary

  • Use sentinel and structured errors when needed

  • fmt.Errorf and errors.New otherwise

  • Match with errors.Is and errors.As to handle

Conclusion

Don’t

  • export until necessary

  • fire-and-forget goroutines

  • noisy error messages

Do

  • plan for expansion

  • control goroutine lifetimes

  • expose and handle errors

Discussion

gopher dance long 3x
gopher dance long 3x

Gopher image credits: egonelbre/gophers