We will use a series to explain the complete practice of microservices from requirements to online, from code to k8s deployment, from logging to monitoring.
The whole project uses the micro services developed by go zero, which basically includes go zero and some middleware developed by relevant go zero authors. The technology stack used is basically the self-developed component of the go zero project team, which is basically the go zero software.
Actual project address: https://github.com/Mikaelemmmm/go-zero-looklook
1. Overview
During our normal development, when the program makes an error, we hope to quickly locate the problem through the error log (then the parameters passed in, including stack information, must be printed to the log), but at the same time, we want to return to the front-end user-friendly and understandable error prompt. If these two points are only through one FMT Error,errors. If new returns an error message, it is definitely impossible to do so, unless the log is recorded at the same time when it returns the error prompt at the front end. In this way, the log will fly all over the sky, the code will be ugly, and the log will be ugly at that time.
Let's think about it. If there is a unified place to record the log, and only one return err is needed in the business code, the error prompt information and log information returned to the front end can be separated from the prompt and record. If it is implemented according to this idea, it's not too cool. Yes, go zero look is handled in this way. Let's take a look.
2. rpc error handling
Under normal circumstances, go zero's rpc service is based on grpc, and the default error returned is grpc's status Error cannot merge our customized errors, and it is not suitable for our customized errors. Its error code and error type are defined to die in the grpc package. ok, if we can use the custom error to return in rpc, then it will be converted to the status of grpc when the interceptor returns uniformly Error, can our rpc err and api err manage our own errors in a unified way?
Let's take a look at the status of grpc What's in the code of error
package codes // import "google.golang.org/grpc/codes"
import (
"fmt"
"strconv"
)
// A Code is an unsigned 32-bit error code as defined in the gRPC spec.
type Code uint32
.......
The error code corresponding to the err of grpc is actually a uint32. We define the error ourselves, use uint32, and then turn it into the err of grpc when the global interceptor of rpc returns
So we define the global error code in app/common/xerr
errCode.go
package xerr // Successful return const OK uint32 = 200 // The first three represent business and the last three represent specific functions // Global error code const SERVER_COMMON_ERROR uint32 = 100001 const REUQES_PARAM_ERROR uint32 = 100002 const TOKEN_EXPIRE_ERROR uint32 = 100003 const TOKEN_GENERATE_ERROR uint32 = 100004 const DB_ERROR uint32 = 100005 // User module
errMsg.go
package xerr
var message map[uint32]string
func init() {
message = make(map[uint32]string)
message[OK] = "SUCCESS"
message[SERVER_COMMON_ERROR] = "The server is wandering,Try again later"
message[REUQES_PARAM_ERROR] = "Parameter error"
message[TOKEN_EXPIRE_ERROR] = "token Invalid, please log in again"
message[TOKEN_GENERATE_ERROR] = "generate token fail"
message[DB_ERROR] = "Database busy,Please try again later"
}
func MapErrMsg(errcode uint32) string {
if msg, ok := message[errcode]; ok {
return msg
} else {
return "The server is wandering,Try again later"
}
}
func IsCodeErr(errcode uint32) bool {
if _, ok := message[errcode]; ok {
return true
} else {
return false
}
}
errors.go
package xerr
import "fmt"
// Common fixed error
type CodeError struct {
errCode uint32
errMsg string
}
// Error code returned to the front end
func (e *CodeError) GetErrCode() uint32 {
return e.errCode
}
// Return to the front-end display end error message
func (e *CodeError) GetErrMsg() string {
return e.errMsg
}
func (e *CodeError) Error() string {
return fmt.Sprintf("ErrCode:%d，ErrMsg:%s", e.errCode, e.errMsg)
}
func NewErrCodeMsg(errCode uint32, errMsg string) *CodeError {
return &CodeError{errCode: errCode, errMsg: errMsg}
}
func NewErrCode(errCode uint32) *CodeError {
return &CodeError{errCode: errCode, errMsg: MapErrMsg(errCode)}
}
func NewErrMsg(errMsg string) *CodeError {
return &CodeError{errCode: SERVER_COMMON_ERROR, errMsg: errMsg}
}
For example, we use rpc code when registering users
package logic
import (
"context"
"looklook/app/identity/cmd/rpc/identity"
"looklook/app/usercenter/cmd/rpc/internal/svc"
"looklook/app/usercenter/cmd/rpc/usercenter"
"looklook/app/usercenter/model"
"looklook/common/xerr"
"github.com/pkg/errors"
"github.com/tal-tech/go-zero/core/logx"
"github.com/tal-tech/go-zero/core/stores/sqlx"
)
var ErrUserAlreadyRegisterError = xerr.NewErrMsg("The user is already registered")
type RegisterLogic struct {
ctx context.Context
svcCtx *svc.ServiceContext
logx.Logger
}
func NewRegisterLogic(ctx context.Context, svcCtx *svc.ServiceContext) *RegisterLogic {
return &RegisterLogic{
ctx: ctx,
svcCtx: svcCtx,
Logger: logx.WithContext(ctx),
}
}
func (l *RegisterLogic) Register(in *usercenter.RegisterReq) (*usercenter.RegisterResp, error) {
user, err := l.svcCtx.UserModel.FindOneByMobile(in.Mobile)
if err != nil && err != model.ErrNotFound {
return nil, errors.Wrapf(xerr.ErrDBError, "mobile:%s,err:%v", in.Mobile, err)
}
if user != nil {
return nil, errors.Wrapf(ErrUserAlreadyRegisterError, "User already exists mobile:%s,err:%v", in.Mobile, err)
}
var userId int64
if err := l.svcCtx.UserModel.Trans(func(session sqlx.Session) error {
user := new(model.User)
user.Mobile = in.Mobile
user.Nickname = in.Nickname
insertResult, err := l.svcCtx.UserModel.Insert(session, user)
if err != nil {
return errors.Wrapf(xerr.ErrDBError, "err:%v,user:%+v", err, user)
}
lastId, err := insertResult.LastInsertId()
if err != nil {
return errors.Wrapf(xerr.ErrDBError, "insertResult.LastInsertId err:%v,user:%+v", err, user)
}
userId = lastId
userAuth := new(model.UserAuth)
userAuth.UserId = lastId
userAuth.AuthKey = in.AuthKey
userAuth.AuthType = in.AuthType
if _, err := l.svcCtx.UserAuthModel.Insert(session, userAuth); err != nil {
return errors.Wrapf(xerr.ErrDBError, "err:%v,userAuth:%v", err, userAuth)
}
return nil
}); err != nil {
return nil, err
}
// 2. Generate token
resp, err := l.svcCtx.IdentityRpc.GenerateToken(l.ctx, &identity.GenerateTokenReq{
UserId: userId,
})
if err != nil {
return nil, errors.Wrapf(ErrGenerateTokenError, "IdentityRpc.GenerateToken userId : %d , err:%+v", userId, err)
}
return &usercenter.RegisterResp{
AccessToken: resp.AccessToken,
AccessExpire: resp.AccessExpire,
RefreshAfter: resp.RefreshAfter,
}, nil
}
errors.Wrapf(ErrUserAlreadyRegisterError, "User already exists mobile:%s,err:%v", in.Mobile, err)
Here we use the go default errors package errors Wrapf (if you don't understand here, check Wrap and wrapf under go's errors package)
The first parameter, ErrUserAlreadyRegisterError, is defined above using xerr Newerrmsg ("the user has been registered") returns a friendly prompt to the front end. Remember that the method under our xerr package is used here
The second parameter is recorded in the server log, which can be written in detail. It doesn't matter at all. It will only be recorded in the server and will not be returned to the front end
Let's see why the first parameter can be returned to the front end, and the second parameter is to record the log
⚠️ [note] in the main method of rpc startup file, we add the global interceptor of grpc, which is very important. If we don't add it, we can't realize it
package main
......
func main() {
........
//RPC log, global interceptor of grpc
s.AddUnaryInterceptors(rpcserver.LoggerInterceptor)
.......
}
Let's look at rpcserver Implementation of loggerinterceptor
import (
...
"github.com/pkg/errors"
)
func LoggerInterceptor(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (resp interface{}, err error) {
resp, err = handler(ctx, req)
if err != nil {
causeErr := errors.Cause(err) // err type
if e, ok := causeErr.(*xerr.CodeError); ok { //Custom error type
logx.WithContext(ctx).Errorf("[RPC-SRV-ERR] %+v", err)
//Convert to grpc err
err = status.Error(codes.Code(e.GetErrCode()), e.GetErrMsg())
} else {
logx.WithContext(ctx).Errorf("[RPC-SRV-ERR] %+v", err)
}
}
return resp, err
}
When a request enters the rpc service, first enter the interceptor and then execute the handler method. If you want to handle something before entering, you can write it before the handler method. What we want to deal with is the return result. If there is an error, so we use GitHub under the handler COM / PKG / errors package. This package is often used in go to handle errors. This is not the official errors package, but it is well designed. The official Wrap and Wrapf of go draw on the ideas of this package.
Because when our grpc internal business returns an error
1) if it is our own business error, we will uniformly generate the error with xerr, so we can get the error information defined by us. Because our own error is also uint32, it is uniformly converted to grpc error err = status Error (codes. Code (e.GetErrCode()), e.GetErrMsg()), then the obtained here, e.GetErrCode() is the code we defined, and e.GetErrMsg() is the second parameter of the error we defined earlier
2) However, there is another case where the rpc service is abnormal. The error thrown out at the bottom is a grpc error. In this case, we can just record the exception directly
3. api error
When api called Register in rpc in logic, rpc returned the error message code for the above second steps as follows
......
func (l *RegisterLogic) Register(req types.RegisterReq) (*types.RegisterResp, error) {
registerResp, err := l.svcCtx.UsercenterRpc.Register(l.ctx, &usercenter.RegisterReq{
Mobile: req.Mobile,
Nickname: req.Nickname,
AuthKey: req.Mobile,
AuthType: model.UserAuthTypeSystem,
})
if err != nil {
return nil, errors.Wrapf(err, "req: %+v", req)
}
var resp types.RegisterResp
_ = copier.Copy(&resp, registerResp)
return &resp, nil
}
Here is also errors. Using the standard package Wrapf, that is, all errors returned in our business are applicable to the errors of the standard package, but the internal parameters should use the errors defined by xerr
Here are two points to note
1) For example, if the api does not want to handle the error, it will directly return the error message to the front end (if we do not want to handle it, we will directly return the error message to the rpc)
In this case, just write it directly like the above figure, and use err of rpc call directly as errors The first parameter of wrapf is thrown out, but the second parameter is better to record the detailed log you need for later viewing in the api log
2) api service no matter what error message rpc returns, I want to redefine myself and return the error message to the front end (for example, rpc has returned "user already exists". When api wants to call rpc, as long as there is an error, I will return to the front end "user registration failed")
In this case, it can be written as follows (of course, you can put xerr.NewErrMsg("user registration failed") on the top of the code to use a variable, or put a variable here)
func (l *RegisterLogic) Register(req types.RegisterReq) (*types.RegisterResp, error) {
.......
if err != nil {
return nil, errors.Wrapf(xerr.NewErrMsg("User registration failed"), "req: %+v,rpc err:%+v", req,err)
}
.....
}
Next, let's look at how to deal with the final return to the front end. We then look at app / usercenter / CMD / API / internal / handler / user / registerhandler go
func RegisterHandler(ctx *svc.ServiceContext) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
var req types.RegisterReq
if err := httpx.Parse(r, &req); err != nil {
httpx.Error(w, err)
return
}
l := user.NewRegisterLogic(r.Context(), ctx)
resp, err := l.Register(req)
result.HttpResult(r, w, resp, err)
}
}
It can be seen here that the handler code generated by go zero look is different from the code generated by the default official goctl in two places, that is, when dealing with error handling, we replace it with our own error handling, in common / result / httpresult go
[note] some people will say that every time we use goctl, we have to change it manually. That's not a lot of trouble. Here we use the template function provided by go zero (we need to learn from the official documents if we don't know this yet). Just modify the template generated by the handler. The template file of the whole project is placed under deploy/goctl, Here, the template modified by hanlder is in deploy/goctl / 1.2.3-cli / API / handler tpl
ParamErrorResult is very simple and is dedicated to handling parameter errors
// http parameter error return
func ParamErrorResult(r *http.Request, w http.ResponseWriter, err error) {
errMsg := fmt.Sprintf("%s ,%s", xerr.MapErrMsg(xerr.REUQES_PARAM_ERROR), err.Error())
httpx.WriteJson(w, http.StatusBadRequest, Error(xerr.REUQES_PARAM_ERROR, errMsg))
}
Let's focus on HttpResult, the error handling method returned by the business
// http return
func HttpResult(r *http.Request, w http.ResponseWriter, resp interface{}, err error) {
if err == nil {
// Successful return
r := Success(resp)
httpx.WriteJson(w, http.StatusOK, r)
} else {
// Error return
errcode := xerr.SERVER_COMMON_ERROR
errmsg := "The server is running away. I'll try again later"
causeErr := errors.Cause(err) // err type
if e, ok := causeErr.(*xerr.CodeError); ok {
// Custom error type
// Custom CodeError
errcode = e.GetErrCode()
errmsg = e.GetErrMsg()
} else {
if gstatus, ok := status.FromError(causeErr); ok {
// grpc err error
grpcCode := uint32(gstatus.Code())
if xerr.IsCodeErr(grpcCode) {
// Distinguish between user-defined errors and system bottom layer and db errors. Bottom layer and db errors cannot be returned to the front end
errcode = grpcCode
errmsg = gstatus.Message()
}
}
}
logx.WithContext(r.Context()).Errorf("[API-ERR] : %+v ", err)
httpx.WriteJson(w, http.StatusBadRequest, Error(errcode, errmsg))
}
}
err: log error to log
errcode: the error code returned to the front end
errmsg: friendly error message returned to the front end
Success is returned directly. If an error is encountered, GitHub is also used COM / PKG / errors package to determine whether the error is our own defined error (the error defined in the api directly uses our own defined xerr) or grpc error (thrown out by rpc service). If it is grpc error, it is converted into our own error code through uint32, According to the error code, find the defined error information in the error information defined by ourselves and return it to the front end. If the api error is directly returned to the error information defined by ourselves and can't be found, then return the default error "the server is out of service",
4. Ending
Here, the error handling message has been clearly described. Next, we need to see how to collect and view the error log printed on the server, which involves the log collection system.
Project address
https://github.com/zeromicro/go-zero
https://gitee.com/kevwan/go-zero
Welcome to go zero and star support us!
Wechat communication group
Focus on the "micro service practice" official account and click on the exchange group to get the community community's two-dimensional code.