java multithreaded Callable -- segment a large list and merge the results

Posted by l9pt5 on Wed, 24 Jun 2020 07:20:34 +0200

Code reference http://bbs.csdn.net/topics/391070227?page=1    

Here is the posted Code:

public void dealListWithMutiThread(){
        List<Object> list = new ArrayList<Object>(10000);
        int index = 0;
        ExecutorService ex = Executors.newFixedThreadPool(5);
        int dealSize = 2000;
        List<Future<List<Object>>> futures = new ArrayList<>(5);
                //distribution
        for(int i=0;i<= 5;i++,index+=dealSize){
            int start = index;
            if(start>=list.size()) break;
            int end = start + dealSize;
            end = end>list.size() ? list.size() : end;
            futures.add(ex.submit(new Task(list,start,end)));
        }
        try {
            //handle
            List<Object>  result = new ArrayList<>();
            for(Future<List<Object>> future : futures){
                //Merge operation
                result.addAll(future.get());
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
     
    private class Task implements Callable<List<Object>>{
         
        private List<Object> list;
        private int start;
        private int end;
         
        public Task(List<Object> list,int start,int end){
            this.list = list;
            this.start = start;
            this.end = end;
        }
 
        @Override
        public List<Object> call() throws Exception {
            Object obj = null;
            List<Object> retList = new ArrayList<Object>();
            for(int i=start;i<end;i++){
                obj = list.get(i);
                //Your processing logic
            }
            //Return processing results
            return retList;
        }
    }

Note: must multithreading be faster than a single thread?

The purpose of multi-threaded programming is to "maximize the use of cpu resources". When a thread's processing does not need to occupy cpu but only deals with I / O, OEM IOS and other resources, other threads that need to occupy cpu resources will have the opportunity to obtain cpu resources. So when I open a single thread, the cpu utilization may not reach 100%, but when I open multiple threads, the cpu utilization will often reach 100%;

Multithreading and multiprocessing

If multi process is adopted, it is necessary to switch the memory needed by the process to which the thread belongs. This time cost is a lot. The cost of thread switching is very small. Threads can Shared memory Of. Therefore, it costs much less to adopt multithreading than multiprocessing.

However, thread switching still needs time consumption, so it takes a process with two threads to execute more time than a process with one thread to execute twice. That is, multithreading will not improve the execution speed of the program, but will reduce the speed, but for users, it can reduce the response time of users.

To sum up: when it comes to IO and other operations that do not occupy the cpu all the time, multithreading can improve the efficiency; however, if it is only a large number of calculations, the efficiency of multithreading will be significantly lower than that of single thread;

Next, a section of code using ThreadLocal is pasted in;



Source: http://blog.csdn.net/hardworking0323/article/details/52266325
/* 
 ThreadLocal 
 It is a container for storing thread local variables. I think it should be called thread local variable, 
 I don't understand why Sun engineers were named this way. 
 Back in the era of JDK 1.2, java.lang.ThreadLocal  It was born, 
 It is designed to solve the problem of multithreading concurrency, but it is a little difficult to use, so it has not been widely used 
 It can solve the problem of synchronous preemption of resources 
 ThreadLocal It is not used to solve the problem of object sharing access, but mainly provides a thread to keep the object method and a convenient object access method to avoid parameter passing  
 ThreadLocal The most suitable application is to access the object based on multiple instances of threads (each thread corresponds to one instance), and this object needs to be used in many places.  
 */  
public class DBUtil {  
    // Database configuration  
    private static final String driver = "com.mysql.jdbc.Driver";  
    private static final String url = "jdbc:mysql://localhost:3306/test";  
    private static final String username = "root";  
    private static final String password = "root";  
  
    // Define a local thread variable to place the database connection (so that each thread has its own connection)  
    private static ThreadLocal<Connection> connContainer = new ThreadLocal<Connection>();  
  
    // Get connection  
    public static Connection getConnection() {  
        Connection conn = connContainer.get();  
        try {  
            if (conn == ) {  
                Class.forName(driver);  
                conn = DriverManager.getConnection(url, username, password);  
            }  
        } catch (Exception e) {  
            e.printStackTrace();  
        } finally {  
            connContainer.set(conn);  
        }  
        return conn;  
    }  
  
    // Close connection  
    public static void closeConnection() {  
        Connection conn = connContainer.get();  
        try {  
            if (conn != ) {  
                conn.close();  
            }  
        } catch (Exception e) {  
            e.printStackTrace();  
        } finally {  
            connContainer.remove();  
        }  
    }  
}  

Topics: MySQL Database JDBC Programming