Maven typed in local and remote libraries, used in ecpliseidea

Posted by lostcarpark on Wed, 03 Jul 2019 18:29:10 +0200

Other blogger posts: http://blog.csdn.net/dhmpgt/article/details/9998321

Maven

Tools for managing project jar packages automatically download the corresponding jar packages from the server library to the project based on the configuration file.

Prepare the file:

install

1) Unzip repository to D:/maven/repositoryserver as Maven local server Library folder

2) Create folder D:/maven/repositoryclient for Maven local libraries

3) Unzip the nexus package to D:/maven/nexus

a) Enter: D:mavennexusnexus-2.14.2-01binjswwindows-x86-64 Start the registration service install-nexus.bat Service name:nexus

b) Administrator Run: start-nexus.bat Start Service

c) Open page to verify: localhost:8081/nexus User name: Password: admin:admin123

4) Enter: D:\maven\apache-maven-3.1.1conf folder.Configure settings.xml

a) Set the path to the local library: <localRepository>D:\mavenrepositoryclient</localRepository>

b) Set the path of the remote library:

<id>mirrorId</id>

<mirrorOf>*</mirrorOf>[D1]

<name>Human Readable Name for thisMirror.</name>

<url>http://192.168.10.228:8081/nexus/content/groups/public/</url>

</mirror>

The url sources are as follows:

Settings are required on the Maven Web page:

Set Local Library Path for Local Library 3rd part

Adjust the order in which maven searches for jar packages:

Indexes need to be rebuilt to associate local libraries

You can add the address of the remote library you want

Address of the library you can refer to: http://central.maven.org/maven2/.

Cloudera's library: * https://repository.cloudera.com/content/repositories/releases/

Next you need to configure the order in which libraries are loaded: local libraries are first, remote libraries are second

3rd party This sets the location of the local library (I don't know yet between this library and the local private library, but it doesn't affect usage much)

After saving, test:

Indicates that maven is ready for use

Open eclipse to create a project:

New->maven Project

Set up eclipse

Windows->perferences->maven

add the location of the maven client folder after installations

Add maven profile

Click to finish

Use maven

Search for the jar package that needs to be imported from the maven web page, and copy the corresponding attribute parameters into the maven configuration file:

<groupId>org.scala-lang</groupId>

<artifactId>scala-reflect</artifactId>

</dependency>

Once Ctrl+s is saved, the jar package will be automatically downloaded

[D1] The reason for this configuration is unclear

Configure the maven project in idea

Create the project as a maven project and proceed directly to the next step

Then modify the path of the maven Library in file-settings:

Then configure the pom.xml configuration file so that maven automatically downloads the dependent jar packages as follows:

Dependent configuration information can be found on the maven web page and added directly to the pom file

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.deppon.www</groupId>
    <artifactId>TechnicalPreview</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
        <encoding>UTF-8</encoding>
        <scala.version>2.10.5</scala.version>
        <spark.version>1.6.0</spark.version>
        <hadoop.version>2.6.4</hadoop.version>
        <hbase.version>1.2.0-cdh5.7.0</hbase.version>
    </properties>

    <dependencies>
        <!--Spark rely on-->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.10</artifactId>
            <version>${spark.version}</version>
        </dependency>

        <!--hadoop rely on-->
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>${hadoop.version}</version>
        </dependency>

        <!--Add to Hbase rely on-->
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>1.2.0-cdh5.7.0</version>
            <!--<version>0.99.2</version>-->
            <!--<version>0.98.24-hadoop2</version>-->
        </dependency>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-server</artifactId>
            <version>1.2.0-cdh5.7.0</version>
            <!--<version>0.99.2</version>-->
            <!--<version>0.98.24-hadoop2</version>-->
        </dependency>

        <!--Add to Kafka rely on-->
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.10</artifactId>
            <version>0.9.0.0</version>
        </dependency>

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>0.9.0.0</version>
        </dependency>


        <!--Scala rely on-->
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>${scala.version}</version>
        </dependency>

        <!--Add to spark-streaming rely on-->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_2.10</artifactId>
            <!--<version>1.6.1</version>-->
            <version>1.6.0</version>
        </dependency>

        <!--Add to sparkStreaming and kafka Integration Dependency-->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming-kafka_2.10</artifactId>
           <!-- <version>1.6.1</version>-->
            <version>1.6.0</version>
        </dependency>

        <!--zookeeper rely on-->
        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.4.5-cdh5.7.0</version>
        </dependency>

        <!--Import Oracle JDBC rely on-->
        <dependency>
            <groupId>com.oracle</groupId>
            <artifactId>ojdbc6</artifactId>
            <version>11.1.0</version>
        </dependency>

        <!--Import Resolution json Dependency on-->
        <dependency>
            <groupId>net.minidev</groupId>
            <artifactId>json-smart</artifactId>
            <version>1.1.1</version>
        </dependency>

        <!--Import spark-sql rely on-->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.10</artifactId>
            <version>1.6.0</version>
            <!--<version>1.5.2</version>-->
        </dependency>


        <!--Import hive spark rely on-->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-hive_2.10</artifactId>
            <version>1.6.0-cdh5.7.0</version>
        </dependency>




    </dependencies>


    <build>
        <sourceDirectory>src/main/scala</sourceDirectory>
        <testSourceDirectory>src/test/scala</testSourceDirectory>
        <plugins>
            <plugin>
                <groupId>net.alchim31.maven</groupId>
                <artifactId>scala-maven-plugin</artifactId>
                <version>3.2.2</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>testCompile</goal>
                        </goals>
                        <configuration>
                            <args>
                                <!--<arg>-make:transitive</arg>-->
                                <arg>-dependencyfile</arg>
                                <arg>${project.build.directory}/.scala_dependencies</arg>
                            </args>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.4.3</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <filters>
                                <filter>
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>
                                    </excludes>
                                </filter>
                            </filters>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>


</project>

Topics: Maven Spark Apache Scala

Programmer Think