How do I implement distributed ID s using MongoDB+Springboot?

Posted by pdmiller on Wed, 10 Jul 2019 19:17:39 +0200

How do I implement distributed ID s in SpringCloud distributed systems using MongoDB?

1. Background

How to implement distributed IDS and search for relevant data will generally give you these solutions:

  • Use database self-incrementing Id
  • Use reids incr command
  • Use UUID
  • snowflake algorithm for Twitter
  • Generating unique ID s using zookeeper
  • ObjectId of MongoDB

In addition, after crawling the known user id, I found that the known user ID is 32 bits, initially determined that md5 encryption was used, and then converted all to lowercase.See the articles I shared before about how to crawl user information.The technical scheme adopted in this paper is the objectId of mogoodb.

2. How mongodb implements distributed ID s

MongoDB's ObjectId is designed to be lightweight and can be easily generated by different machines in a globally unique and identical way.MongoDB has been designed from the start as a distributed database, and handling multiple nodes is a core requirement.Make it much easier to generate in a fragmented environment.

Its format:


mongo.png
  • The first four bytes are timestamps in seconds starting from the standard era.The timestamp, combined with the next five bytes, provides second-level uniqueness.Because the timestamp is first, this means that the ObjectId is roughly in the order in which it was inserted.This is useful in some areas, such as increasing efficiency as an index.These four bytes also imply when the document was created.Most client class libraries expose a method to get this information from ObjectId.

  • The next 3 bytes are the unique identifiers of the host.This is usually a hash value of the machine host name.This ensures that different hosts generate different ObjectId s without conflict.
    To ensure that the ObjectId produced by multiple processes concurrently on the same machine is unique, the next two bytes come from the process identifier (PID) that generated the ObjectId.

  • The first 9 bytes guarantee that ObjectId s generated by different processes on different machines are unique in the same second.
  • The last 3 bytes is an auto-incrementing counter, ensuring that ObjectIds produced by the same process in the same second are also different.A maximum of 2563 (16 777 216) different ObjectIds per process are allowed in the same second.

3. Coding

Introduce mongodb in springboot:

       <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!-- open web-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>


       <!--mongodb -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>

Create an entity class:

public class Customer {

    @Id
    public String id;

    public String firstName;
    public String lastName;

    public Customer() {}

    public Customer(String firstName, String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    @Override
    public String toString() {
        return String.format(
                "Customer[id=%s, firstName='%s', lastName='%s']",
                id, firstName, lastName);
    }


    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    public String getLastName() {
        return lastName;
    }

    public void setLastName(String lastName) {
        this.lastName = lastName;
    }
}

Create a mongodb interface class:

/**
 * Created by fangzhipeng on 2017/4/1.
 */


public interface CustomerRepository extends MongoRepository<Customer, String> {

    public Customer findByFirstName(String firstName);
    public List<Customer> findByLastName(String lastName);

}

Test class:

 @Autowired
    CustomerRepository customerRepository;


@Test
public void mongodbIdTest(){
Customer customer=new Customer("lxdxil","dd");
        customer=customerRepository.save(customer);
        logger.info( "mongodbId:"+customer.getId());
}

4. References

Accessing Data with MongoDB

MongoDB's Deep ObjectId

MongoDB Tutorial

Topics: MongoDB Spring Database Zookeeper