Nutch2.1+Hbase+Solr to quickly build a crawler and search engine (fast, basically within 2 hours)

Note: This method is for quick experience or small data volume, not suitable for large data volume production environment. Environmental preparation: Centos7 Nutch2.2.1 JAVA1.8 ant1.9.14 HBase 0.90.4 (stand-alone version) solr7.7 Relevant download address: Links: https://pan.baidu.com/s/1Tut2CcKoJ9-G-HBq8zexMQ Extraction code: v75v Start inst ...

Posted by MAXIEDECIMAL on Thu, 26 Sep 2019 07:49:34 +0200

kafka installation and start-up

Kafka's background knowledge has been talked about a lot, let's start to practice now, assuming you don't have Kafka and Zoo Keeper environments. Step 1: Download the code Download version 1.1.0 and unzip it. > tar -xzf kafka_2.12-2.3.0.tgz > cd kafka_2.12-2.3.0 Step 2: Start the service Running kafka requir ...

Posted by Osiris Beato on Wed, 18 Sep 2019 05:49:10 +0200

Simple spring cloud application building

I. Sprcloud Microsoft Service Architecture 2. Building Sprcloud application step by step service client Service: Application that provides services. Port is 808x Client: The application port for invoking the service is 809x The whole program uses the client end to call the server end program according to the user id to return user informatio ...

Posted by nightdesigns on Wed, 11 Sep 2019 12:54:50 +0200

ZooKeeper Series: Logging

Each write operation in Zookeeper maintains a transaction log dedicated to writing znode operations. Only the data changes confirmed by the transaction log will take effect throughout the cluster. 1,TxnLog TxnLog is a write transaction log interface, which mainly includes the following interfaces: rollLog: Log scroll to start a new log ap ...

Posted by jackofalltrades on Mon, 09 Sep 2019 14:17:49 +0200

ZooKeeper Series: Leader Call Chain

Leader receives client requests by starting Leader Zoo Keeper Server. First, we look at the definition of its processing chain. From the source code, we can see that the sequence of the processing chain of LeaderZoo Keeper Server is as follows:   LeaderRequestProcessor: Leader call chain begins PrepRequest Processor: Write action prepar ...

Posted by praxedis on Mon, 09 Sep 2019 08:13:55 +0200

Haoop 3.1.2 HA High Availability Buildings

Official Documents https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html#Configuration_details ** 1. Resource Role Planning ** ** 2. Configuration ** 2.1 Backup of fully distributed cluster. If you ...

Posted by ONiX on Thu, 05 Sep 2019 09:08:19 +0200

Dubbo Operating Principle

I. SOA PatternsA brief introduction to the SOA pattern is helpful for us to understand Dubbo later. What is the SOA model? SQA (Service-Oriented Architecture) is a service-oriented architecture that splits different functional units of an application (understood here as services). Under this architecture, the project does not interact directly ...

Posted by bslevin on Mon, 26 Aug 2019 08:23:42 +0200

kafka distributed cluster construction

Version 1 CentOS 7.5 zookeeper-3.4.12 kafka _2.12-1.1.0 II. zookeeper Installation 1. Download and decompress zookeeper compression package   tar -zvxf zookeeper-3.4.12.tar.gz   2. Creating Data and Log Folders mkdir /usr/local/zookeeper-3.4.12/data mkdir /usr/local/zookeeper-3.4.12/logs 3. Copy configuration files Go to th ...

Posted by justAnoob on Sat, 24 Aug 2019 11:49:00 +0200

Generate SSL certificates and configure SSL authentication for Kafka

1. Generating relevant SSL certificates Relevant knowledge points: Java SSL authentication: SSL (Secure Socket Layer Secure Socket Layer) and its successor Transport Layer Security (TLS) are security protocols that provide security and data integ ...

Posted by evdawg80 on Tue, 13 Aug 2019 15:47:51 +0200

An Overview of zookeeper

In the past, because of the business needs of the company, some knowledge points of zookeeper were sorted out and shared. For some small partners who have just come into contact with zookeeper, maybe we can learn something from it. Introduction to zookeeper brief introduction Zookeeper is committed to providing a distributed coordination servic ...

Posted by neylitalo on Sun, 11 Aug 2019 17:12:54 +0200