Nutch2.1+Hbase+Solr to quickly build a crawler and search engine (fast, basically within 2 hours)
Note: This method is for quick experience or small data volume, not suitable for large data volume production environment.
Environmental preparation:
Centos7
Nutch2.2.1
JAVA1.8
ant1.9.14
HBase 0.90.4 (stand-alone version)
solr7.7
Relevant download address:
Links: https://pan.baidu.com/s/1Tut2CcKoJ9-G-HBq8zexMQ Extraction code: v75v
Start inst ...
Posted by MAXIEDECIMAL on Thu, 26 Sep 2019 07:49:34 +0200
kafka installation and start-up
Kafka's background knowledge has been talked about a lot, let's start to practice now, assuming you don't have Kafka and Zoo Keeper environments.
Step 1: Download the code
Download version 1.1.0 and unzip it.
> tar -xzf kafka_2.12-2.3.0.tgz
> cd kafka_2.12-2.3.0
Step 2: Start the service
Running kafka requir ...
Posted by Osiris Beato on Wed, 18 Sep 2019 05:49:10 +0200
Simple spring cloud application building
I. Sprcloud Microsoft Service Architecture
2. Building Sprcloud application step by step
service client
Service: Application that provides services. Port is 808x
Client: The application port for invoking the service is 809x
The whole program uses the client end to call the server end program according to the user id to return user informatio ...
Posted by nightdesigns on Wed, 11 Sep 2019 12:54:50 +0200
ZooKeeper Series: Logging
Each write operation in Zookeeper maintains a transaction log dedicated to writing znode operations. Only the data changes confirmed by the transaction log will take effect throughout the cluster.
1,TxnLog
TxnLog is a write transaction log interface, which mainly includes the following interfaces:
rollLog: Log scroll to start a new log
ap ...
Posted by jackofalltrades on Mon, 09 Sep 2019 14:17:49 +0200
ZooKeeper Series: Leader Call Chain
Leader receives client requests by starting Leader Zoo Keeper Server.
First, we look at the definition of its processing chain. From the source code, we can see that the sequence of the processing chain of LeaderZoo Keeper Server is as follows:
LeaderRequestProcessor: Leader call chain begins
PrepRequest Processor: Write action prepar ...
Posted by praxedis on Mon, 09 Sep 2019 08:13:55 +0200
Haoop 3.1.2 HA High Availability Buildings
Official Documents
https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html#Configuration_details
**
1. Resource Role Planning
**
**
2. Configuration
**
2.1 Backup of fully distributed cluster. If you ...
Posted by ONiX on Thu, 05 Sep 2019 09:08:19 +0200
Dubbo Operating Principle
I. SOA PatternsA brief introduction to the SOA pattern is helpful for us to understand Dubbo later.
What is the SOA model?
SQA (Service-Oriented Architecture) is a service-oriented architecture that splits different functional units of an application (understood here as services). Under this architecture, the project does not interact directly ...
Posted by bslevin on Mon, 26 Aug 2019 08:23:42 +0200
kafka distributed cluster construction
Version 1
CentOS 7.5
zookeeper-3.4.12
kafka _2.12-1.1.0
II. zookeeper Installation
1. Download and decompress zookeeper compression package
tar -zvxf zookeeper-3.4.12.tar.gz
2. Creating Data and Log Folders
mkdir /usr/local/zookeeper-3.4.12/data
mkdir /usr/local/zookeeper-3.4.12/logs
3. Copy configuration files
Go to th ...
Posted by justAnoob on Sat, 24 Aug 2019 11:49:00 +0200
Generate SSL certificates and configure SSL authentication for Kafka
1. Generating relevant SSL certificates
Relevant knowledge points:
Java SSL authentication: SSL (Secure Socket Layer Secure Socket Layer) and its successor Transport Layer Security (TLS) are security protocols that provide security and data integ ...
Posted by evdawg80 on Tue, 13 Aug 2019 15:47:51 +0200
An Overview of zookeeper
In the past, because of the business needs of the company, some knowledge points of zookeeper were sorted out and shared. For some small partners who have just come into contact with zookeeper, maybe we can learn something from it.
Introduction to zookeeper
brief introduction Zookeeper is committed to providing a distributed coordination servic ...
Posted by neylitalo on Sun, 11 Aug 2019 17:12:54 +0200