Realization motivation

The simplification and integration of configuration are the two main problems that line expressions want to solve.

In the cumbersome data fragmentation rule configuration, with the increase of data nodes, a large number of repeated configurations make the configuration itself difficult to maintain. The line expression can effectively simplify the workload of data node configuration.

For common sharding algorithms, the implementation of Java code is not conducive to the unified management of configuration. Through the line expression writing fragment algorithm, the rule configuration can be effectively stored together, which is easier to browse and store.

Syntax description

The use of line expressions is very intuitive. You only need to use ${expression} or $- > {expression} to identify line expressions in the configuration. At present, it supports the configuration of data nodes and fragmentation algorithm. The content of the line expression uses Groovy's syntax. All operations that Groovy can support can be supported by the line expression. For example:

${begin..end} indicates the range

${[unit1, unit2, unit_x]} indicates the enumeration value

If there are consecutive ${expression} or $- > {expression} expressions in the line expression, the final result of the whole expression will be Cartesian combined according to the result of each sub expression.

For example, in the following line:

${['online', 'offline']}_table${1..3}

It will eventually resolve to:

online_table1, online_table2, online_table3, offline_table1, offline_table2, offline_table3

to configure

Data node

For evenly distributed data nodes, if the data structure is as follows:

db0
  ├── t_order0
  └── t_order1
db1
  ├── t_order0
  └── t_order1

The line expression can be simplified to:

db${0..1}.t_order${0..1}

perhaps

db$->{0..1}.t_order$->{0..1}

For user-defined data nodes, if the data structure is as follows:

db0
  ├── t_order0
  └── t_order1
db1
  ├── t_order2
  ├── t_order3
  └── t_order4

The line expression can be simplified to:

db0.t_order${0..1},db1.t_order${2..4}

perhaps

db0.t_order$->{0..1},db1.t_order$->{2..4}

For data nodes with prefixes, they can also be flexibly configured through row expressions. If the data structure is as follows:

db0
  ├── t_order_00
  ├── t_order_01
  ├── t_order_02
  ├── t_order_03
  ├── t_order_04
  ├── t_order_05
  ├── t_order_06
  ├── t_order_07
  ├── t_order_08
  ├── t_order_09
  ├── t_order_10
  ├── t_order_11
  ├── t_order_12
  ├── t_order_13
  ├── t_order_14
  ├── t_order_15
  ├── t_order_16
  ├── t_order_17
  ├── t_order_18
  ├── t_order_19
  └── t_order_20
db1
  ├── t_order_00
  ├── t_order_01
  ├── t_order_02
  ├── t_order_03
  ├── t_order_04
  ├── t_order_05
  ├── t_order_06
  ├── t_order_07
  ├── t_order_08
  ├── t_order_09
  ├── t_order_10
  ├── t_order_11
  ├── t_order_12
  ├── t_order_13
  ├── t_order_14
  ├── t_order_15
  ├── t_order_16
  ├── t_order_17
  ├── t_order_18
  ├── t_order_19
  └── t_order_20

The method of separate configuration can be used. First configure the data nodes with prefixes, then configure the data nodes without prefixes, and then automatically combine them by using the Cartesian product of row expressions. The above example can be simplified to:

db${0..1}.t_order_0${0..9}, db${0..1}.t_order_${10..20}

perhaps

db$->{0..1}.t_order_0$->{0..9}, db$->{0..1}.t_order_$->{10..20}

Slicing algorithm

For SQL that has only one partition key and uses = and ＾ IN ＾ to partition, you can use line expression instead of coding method configuration.

The expression inside the row expression is essentially a piece of Groovy code, which can return the corresponding real data source or real table name according to the calculation method of fragment key.

For example, it is divided into 10 libraries. The data with mantissa 0 is routed to the data source with suffix 0, the data with mantissa 1 is routed to the data source with suffix 1, and so on. The line expression used to represent the slicing algorithm is:

ds${id % 10}

perhaps

ds$->{id % 10}