brief introduction

cron A library for managing timing tasks, using Go to achieve the effect of crontab command in Linux. We introduced a similar Go library before—— gron . gron code is small, which is better for learning. But its function is relatively simple, and it is no longer maintained. If there is a need for timed tasks, cron is still recommended.

Quick use

The text code uses Go Modules.

Create directory and initialize:

$ mkdir cron && cd cron
$ go mod init github.com/darjun/go-daily-lib/cron

Install cron. The latest stable version is v3:

$ go get -u github.com/robfig/cron/v3

use:

package main

import (
  "fmt"
  "time"

  "github.com/robfig/cron/v3"
)

func main() {
  c := cron.New()

  c.AddFunc("@every 1s", func() {
    fmt.Println("tick every 1 second")
  })

  c.Start()
  time.Sleep(time.Second * 5)
}

Very simple to use, create a cron object, which is used to manage scheduled tasks.

Call the AddFunc() method of the cron object to add a timer task to the manager. AddFunc() takes two parameters. Parameter 1 specifies the trigger time rule in the form of string. Parameter 2 is a function without parameters, which is called every time it is triggered. @every 1s is triggered once per second, and @ every is followed by a time interval to indicate how often. For example, @ every 1h means every hour, and @ every 1m2s means every 1 minute and 2 seconds. time.ParseDuration() all supported formats can be used here.

Call c.Start() to start the timing cycle.

Note that since c.Start() starts a new goroutine for loop detection, we add a line at the end of the code time.Sleep ( time.Second *5) prevent the main goroutine from exiting.

Running effect, output a line of string every 1s:

$ go run main.go 
tick every 1 second
tick every 1 second
tick every 1 second
tick every 1 second
tick every 1 second

Time format

Similar to the crontab command in Linux, the cron library supports five space separated fields to represent time. The meanings of these five fields are as follows:

• Minutes: minutes, value range [0-59], support special characters * /, -;
• Hours: hour, value range [0-23], support special characters * /, -;
• Day of month: the day of the month, value range [1-31], support special characters * /, -?;
• Month: month, value range [1-12] or month name abbreviation [JAN-DEC], support special characters * /, -;
• Day of week: week calendar, value range [0-6] or abbreviation [JUN-SAT], support special characters * /, -?.

Note that both month and week calendar names are case insensitive, meaning SUN/Sun/sun means the same (both Sunday).

The meaning of special characters is as follows:

• *: fields using * can match any value. For example, set the month field (4th) to *, which means every month;
• /: used to specify the step size of the range, for example, setting the small time domain (the second one) to 3-59 / 15 means the third minute is triggered, and then every 15 minutes, so the second trigger is the 18th minute, and the third trigger is 33 minutes... Until the minute is greater than 59;
• ,: used to list some discrete values and multiple ranges. For example, set the domain (5th) of the weekly calendar to MON,WED,FRI for Monday, Wednesday, and Friday;
• -: used to represent the range, for example, setting the domain of hours (the first one) to 9-17 to represent 9 a.m. to 17 p.m. (including 9 and 17);
• ?: it can only be used in the fields of monthly and weekly calendars instead of *, indicating any day of each month / week.

Once we understand the rules, we can define any time:

• 30 * *: minute field is 30, and other fields are * to indicate any. 30 minutes per hour;
• 30 3-6,20-23 * *: the minute domain is 30, and 3-6,20-23 in the hour domain represent 3 to 6 and 20 to 23. 30 minutes at 3,4,5,6,20,21,22,23;
• 0 0 1 1 *: 0 (2nd) on the 1st (3rd) of the 1st (4th) month 1 (3rd) is triggered at 0 (1st) minute.

It's easy to master the format after memorizing the order of these fields and practicing a few more times. Once you are familiar with the rules, you will be able to use the crontab command skillfully.

func main() {
  c := cron.New()

  c.AddFunc("30 * * * *", func() {
    fmt.Println("Every hour on the half hour")
  })

  c.AddFunc("30 3-6,20-23 * * *", func() {
    fmt.Println("On the half hour of 3-6am, 8-11pm")
  })

  c.AddFunc("0 0 1 1 *", func() {
    fmt.Println("Jun 1 every year")
  })

  c.Start()

  for {
    time.Sleep(time.Second)
  }
}

Predefined time rules

For convenience, cron predefines some time rules:

• @yearly: you can also write @ annually, which means 0 o'clock on the first day of each year. Equivalent to 0 01 1 *;
• @monthly: zero on the first day of each month. Equivalent to 0.01 *;
• @weekly: it means 0 o'clock on the first day of the week. Note that the first day is Sunday, that is, the 0 o'clock that ends on Saturday and begins on Sunday. Equivalent to 0 0 * 0;
• @daily: you can also write @ midnight, which means 0 o'clock every day. Equivalent to 0 0 * *;
• @hourly: the beginning of every hour. Equivalent to 0 *. *.

For example:

func main() {
  c := cron.New()

  c.AddFunc("@hourly", func() {
    fmt.Println("Every hour")
  })

  c.AddFunc("@daily", func() {
    fmt.Println("Every day on midnight")
  })

  c.AddFunc("@weekly", func() {
    fmt.Println("Every week")
  })

  c.Start()

  for {
    time.Sleep(time.Second)
  }
}

The above code is just a demonstration of usage. It may take a very long time for the actual operation to have output.

Fixed time interval

cron supports fixed time intervals in the following format:

@every <duration>

It means triggered every duration. < duration > will call time.ParseDuration() function resolution, so the format supported by parseduration can be used. For example, 1h30m10s. In the quick start section, we have demonstrated the use of @ every, which will not be covered here.

time zone

By default, all times are based on the current time zone. Of course, we can also specify the time zone in two ways:

• Add a cron before the time string_ TZ = + specific time zone. The format of specific time zone is before carbon There is a detailed introduction in the article. Asia/Tokyo in Tokyo and America / new in New York_ York；
• Add a time zone option when creating cron objects cron.WithLocation(location), location is time.LoadLocation(zone) the time zone object loaded. Zone is the specific time zone format. Or call the SetLocation() method of the created cron object to set the time zone.

Example:

func main() {
  nyc, _ := time.LoadLocation("America/New_York")
  c := cron.New(cron.WithLocation(nyc))
  c.AddFunc("0 6 * * ?", func() {
    fmt.Println("Every 6 o'clock at New York")
  })

  c.AddFunc("CRON_TZ=Asia/Tokyo 0 6 * * ?", func() {
    fmt.Println("Every 6 o'clock at Tokyo")
  })

  c.Start()

  for {
    time.Sleep(time.Second)
  }
}

Job interface

In addition to directly using parameterless functions as callbacks, cron also supports Job interfaces:

// cron.go
type Job interface {
  Run()
}

We define a structure to implement the interface Job:

type GreetingJob struct {
  Name string
}

func (g GreetingJob) Run() {
  fmt.Println("Hello ", g.Name)
}

Call the AddJob() method of the cron object to add the greeningjob object to the timing Manager:

func main() {
  c := cron.New()
  c.AddJob("@every 1s", GreetingJob{"dj"})
  c.Start()

  time.Sleep(5 * time.Second)
}

Operation effect:

$ go run main.go 
Hello  dj
Hello  dj
Hello  dj
Hello  dj
Hello  dj

The custom structure allows tasks to carry status (Name field).

In fact, the AddJob() method is also called inside the AddFunc() method. First, cron defines a new type FuncJob based on func():

// cron.go
type FuncJob func()

Then let FuncJob implement the Job interface:

// cron.go
func (f FuncJob) Run() {
  f()
}

In the AddFunc() method, the incoming callback is converted to the FuncJob type, and then the AddJob() method is called:

func (c *Cron) AddFunc(spec string, cmd func()) (EntryID, error) {
  return c.AddJob(spec, FuncJob(cmd))
}

Thread safety

cron creates a new goroutine to execute the trigger callback. If these callbacks need to access some resources and data concurrently, we need to explicitly synchronize them.

Custom time format

Cron supports flexible time format. If the default format cannot meet the requirements, we can define our own time format. Time rule string required cron.Parser Object. Let's first look at how the default parser works.

First define each domain:

// parser.go
const (
  Second         ParseOption = 1 << iota
  SecondOptional                        
  Minute                                
  Hour                                  
  Dom                                   
  Month                                 
  Dow                                   
  DowOptional                           
  Descriptor                            
)

Besides Minute/Hour/Dom(Day of month)/Month/Dow(Day of week), Second can also be supported. The relative order is fixed:

// parser.go
var places = []ParseOption{
  Second,
  Minute,
  Hour,
  Dom,
  Month,
  Dow,
}

var defaults = []string{
  "0",
  "0",
  "0",
  "*",
  "*",
  "*",
}

The default time format uses five fields.

We can call cron.NewParser() create your own Parser object and pass in which domains to use in bit format. For example, the following parsers use 6 domains and support Second (seconds):

parser := cron.NewParser(
  cron.Second | cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.Dow | cron.Descriptor,
)

call cron.WithParser(parser) create an option passed in constructor cron.New(), you can specify seconds when using:

c := cron.New(cron.WithParser(parser))
c.AddFunc("1 * * * * *", func () {
  fmt.Println("every 1 second")
})
c.Start()

Here, the time format must use 6 fields in the same order as the const definition above.

Because the above time format is too common, cron defines a convenient function:

// option.go
func WithSeconds() Option {
  return WithParser(NewParser(
    Second | Minute | Hour | Dom | Month | Dow | Descriptor,
  ))
}

Note that descriptors support @ every/@hour, etc. With WithSeconds(), we don't need to create the Parser object manually:

c := cron.New(cron.WithSeconds())

option

The cron object creation uses the option mode. We have introduced three options:

• WithLocation: Specifies the time zone;
• WithParser: use a custom parser;
• WithSeconds: let the time format support seconds. In fact, WithParser is called internally.

cron also offers two other options:

• WithLogger: Custom Logger;
• WithChain: Job wrapper.

WithLogger

WithLogger can set cron to use our customized Logger internally:

func main() {
  c := cron.New(
    cron.WithLogger(
      cron.VerbosePrintfLogger(log.New(os.Stdout, "cron: ", log.LstdFlags))))
  c.AddFunc("@every 1s", func() {
    fmt.Println("hello world")
  })
  c.Start()

  time.Sleep(5 * time.Second)
}

Called above cron.VerbosPrintfLogger() packaging log.Logger , this logger will record the internal scheduling process of cron in detail:

$ go run main.go
cron: 2020/06/26 07:09:14 start
cron: 2020/06/26 07:09:14 schedule, now=2020-06-26T07:09:14+08:00, entry=1, next=2020-06-26T07:09:15+08:00
cron: 2020/06/26 07:09:15 wake, now=2020-06-26T07:09:15+08:00
cron: 2020/06/26 07:09:15 run, now=2020-06-26T07:09:15+08:00, entry=1, next=2020-06-26T07:09:16+08:00
hello world
cron: 2020/06/26 07:09:16 wake, now=2020-06-26T07:09:16+08:00
cron: 2020/06/26 07:09:16 run, now=2020-06-26T07:09:16+08:00, entry=1, next=2020-06-26T07:09:17+08:00
hello world
cron: 2020/06/26 07:09:17 wake, now=2020-06-26T07:09:17+08:00
cron: 2020/06/26 07:09:17 run, now=2020-06-26T07:09:17+08:00, entry=1, next=2020-06-26T07:09:18+08:00
hello world
cron: 2020/06/26 07:09:18 wake, now=2020-06-26T07:09:18+08:00
hello world
cron: 2020/06/26 07:09:18 run, now=2020-06-26T07:09:18+08:00, entry=1, next=2020-06-26T07:09:19+08:00
cron: 2020/06/26 07:09:19 wake, now=2020-06-26T07:09:19+08:00
hello world
cron: 2020/06/26 07:09:19 run, now=2020-06-26T07:09:19+08:00, entry=1, next=2020-06-26T07:09:20+08:0

Let's see what the default Logger looks like:

// logger.go
var DefaultLogger Logger = PrintfLogger(log.New(os.Stdout, "cron: ", log.LstdFlags))

func PrintfLogger(l interface{ Printf(string, ...interface{}) }) Logger {
  return printfLogger{l, false}
}

func VerbosePrintfLogger(l interface{ Printf(string, ...interface{}) }) Logger {
  return printfLogger{l, true}
}

type printfLogger struct {
  logger  interface{ Printf(string, ...interface{}) }
  logInfo bool
}

WithChain

The Job wrapper can add some logic before and after executing the actual Job:

• Capture panic;
• If the last run of Job has not finished, postpone the execution;
• If the last run of Job has not been introduced, skip this execution;
• Record the execution of each Job.

We can compare Chain to middleware of Web processor. In fact, it encapsulates a layer of logic outside the Job's execution logic. Our encapsulation logic needs to be written as a function, passing in a Job type and returning the encapsulated Job. cron defines a type JobWrapper for this function:

// chain.go
type JobWrapper func(Job) Job

Then use a Chain object to combine these jobwrappers:

type Chain struct {
  wrappers []JobWrapper
}

func NewChain(c ...JobWrapper) Chain {
  return Chain{c}
}

Call the Then(job) method of the Chain object to apply these jobwrappers and return the final 'Job:

func (c Chain) Then(j Job) Job {
  for i := range c.wrappers {
    j = c.wrappers[len(c.wrappers)-i-1](j)
  }
  return j
}

Note the order in which jobwrappers are applied.

Built in JobWrapper

cron has built in three more used jobwrappers:

• Recover: capture the panic generated by the internal Job;
• DelayIfStillRunning: when triggered, if the last task has not been completed (taking too long), wait for the last task to be completed before executing;
• Skipfstillrunning: when triggered, if the last task has not been completed, the execution will be skipped.

The following are respectively introduced.

Recover

Let's see how to use it first:

type panicJob struct {
  count int
}

func (p *panicJob) Run() {
  p.count++
  if p.count == 1 {
    panic("oooooooooooooops!!!")
  }

  fmt.Println("hello world")
}

func main() {
  c := cron.New()
  c.AddJob("@every 1s", cron.NewChain(cron.Recover(cron.DefaultLogger)).Then(&panicJob{}))
  c.Start()

  time.Sleep(5 * time.Second)
}

The first time the panicJob is triggered, the panic is triggered. Because of cron.Recover() protection, follow-up tasks can also be performed:

go run main.go 
cron: 2020/06/27 14:02:00 panic, error=oooooooooooooops!!!, stack=...
goroutine 18 [running]:
github.com/robfig/cron/v3.Recover.func1.1.1(0x514ee0, 0xc0000044a0)
        D:/code/golang/pkg/mod/github.com/robfig/cron/v3@v3.0.1/chain.go:45 +0xbc
panic(0x4cf380, 0x513280)
        C:/Go/src/runtime/panic.go:969 +0x174
main.(*panicJob).Run(0xc0000140e8)
        D:/code/golang/src/github.com/darjun/go-daily-lib/cron/recover/main.go:17 +0xba
github.com/robfig/cron/v3.Recover.func1.1()
        D:/code/golang/pkg/mod/github.com/robfig/cron/v3@v3.0.1/chain.go:53 +0x6f
github.com/robfig/cron/v3.FuncJob.Run(0xc000070390)
        D:/code/golang/pkg/mod/github.com/robfig/cron/v3@v3.0.1/cron.go:136 +0x2c
github.com/robfig/cron/v3.(*Cron).startJob.func1(0xc00005c0a0, 0x514d20, 0xc000070390)
        D:/code/golang/pkg/mod/github.com/robfig/cron/v3@v3.0.1/cron.go:312 +0x68
created by github.com/robfig/cron/v3.(*Cron).startJob
        D:/code/golang/pkg/mod/github.com/robfig/cron/v3@v3.0.1/cron.go:310 +0x7a
hello world
hello world
hello world
hello world

Let's see cron.Recover The implementation of () is very simple:

// cron.go
func Recover(logger Logger) JobWrapper {
  return func(j Job) Job {
    return FuncJob(func() {
      defer func() {
        if r := recover(); r != nil {
          const size = 64 << 10
          buf := make([]byte, size)
          buf = buf[:runtime.Stack(buf, false)]
          err, ok := r.(error)
          if !ok {
            err = fmt.Errorf("%v", r)
          }
          logger.Error(err, "panic", "stack", "...\n"+string(buf))
        }
      }()
      j.Run()
    })
  }
}

It is to add the recover() call before executing the job logic of the inner layer. If Job.Run() there is panic during execution. Here, recover () will capture, output the call stack.

DelayIfStillRunning

Let's see how to use it first:

type delayJob struct {
  count int
}

func (d *delayJob) Run() {
  time.Sleep(2 * time.Second)
  d.count++
  log.Printf("%d: hello world\n", d.count)
}

func main() {
  c := cron.New()
  c.AddJob("@every 1s", cron.NewChain(cron.DelayIfStillRunning(cron.DefaultLogger)).Then(&delayJob{}))
  c.Start()

  time.Sleep(10 * time.Second)
}

We added a 2s delay in Run() above, and the output middle interval changed to 2s instead of the scheduled 1s:

$ go run main.go 
2020/06/27 14:11:16 1: hello world
2020/06/27 14:11:18 2: hello world
2020/06/27 14:11:20 3: hello world
2020/06/27 14:11:22 4: hello world

Look at the source code:

// chain.go
func DelayIfStillRunning(logger Logger) JobWrapper {
  return func(j Job) Job {
    var mu sync.Mutex
    return FuncJob(func() {
      start := time.Now()
      mu.Lock()
      defer mu.Unlock()
      if dur := time.Since(start); dur > time.Minute {
        logger.Info("delay", "duration", dur)
      }
      j.Run()
    })
  }
}

First, define a mutual exclusive lock shared by the task sync.Mutex , acquire the lock before each task execution, and release the lock after execution. Therefore, before the end of the previous task, the next task can not acquire the lock successfully, so as to ensure the serial execution of the task.

SkipIfStillRunning

Let's see how to use it first:

type skipJob struct {
  count int32
}

func (d *skipJob) Run() {
  atomic.AddInt32(&d.count, 1)
  log.Printf("%d: hello world\n", d.count)
  if atomic.LoadInt32(&d.count) == 1 {
    time.Sleep(2 * time.Second)
  }
}

func main() {
  c := cron.New()
  c.AddJob("@every 1s", cron.NewChain(cron.SkipIfStillRunning(cron.DefaultLogger)).Then(&skipJob{}))
  c.Start()

  time.Sleep(10 * time.Second)
}

Output:

$ go run main.go
2020/06/27 14:22:07 1: hello world
2020/06/27 14:22:10 2: hello world
2020/06/27 14:22:11 3: hello world
2020/06/27 14:22:12 4: hello world
2020/06/27 14:22:13 5: hello world
2020/06/27 14:22:14 6: hello world
2020/06/27 14:22:15 7: hello world
2020/06/27 14:22:16 8: hello world

Pay attention to the observation time. The difference between the first output and the second output is 3s, because the execution is skipped twice.

Note that there is an essential difference between DelayIfStillRunning and skipfstillrunning. As long as the time of DelayIfStillRunning is long enough, all tasks will be completed step by step. However, the former task may take too long, which may delay the execution of the latter task a little. SkipIfStillRunning skips some executions.

Look at the source code:

func SkipIfStillRunning(logger Logger) JobWrapper {
  return func(j Job) Job {
    var ch = make(chan struct{}, 1)
    ch <- struct{}{}
    return FuncJob(func() {
      select {
      case v := <-ch:
        j.Run()
        ch <- v
      default:
        logger.Info("skip")
      }
    })
  }
}

Define a channel chan struct {} with a cache size of 1 shared by the task. When executing a task, take the value from the channel. If it succeeds, execute it. Otherwise, skip. After execution, send a value to the channel to ensure the next task can be executed. Initially send a value to the channel to ensure the execution of the first task.

summary

cron implementation is relatively small and elegant, and the number of lines of code is not much, it is very worth a look!

If you find a fun and easy-to-use Go language library, you are welcome to submit the issue to GitHub, the daily Go library 😄

reference resources

1. cron GitHub: https://github.com/robfig/cron
2. Go, one library a day, carbon: https://darjun.github.io/2020/02/14/godailylib/carbon/
3. gron of Go daily: https://darjun.github.io/2020/04/20/godailylib/gron/
4. Go daily GitHub: https://github.com/darjun/go-daily-lib

I

My blog: https://darjun.github.io

Welcome to my WeChat official account, GoUpUp, learn together and make progress together.