By Jiefei
A thread pool is a mechanism for managing and reusing threads.
The core idea of a thread pool is to pre-create a certain number of threads and keep them in the pool. When a task needs to be executed, the thread pool takes an idle thread from the pool to execute the task. Once the task is completed, the thread is not destroyed but returned to the thread pool, ready to be used immediately or later for executing other tasks. This mechanism can avoid the performance overhead caused by frequently creating and destroying threads, while also controlling the number of concurrently running threads, thereby improving system performance and resource utilization.
The main components of the thread pool include worker threads, task queues, thread managers, and so on. The design of thread pools helps to optimize the performance and resource utilization of multi-threaded programs, while also simplifying the complexity of thread management and reuse.
• A thread pool can reduce the overhead of thread creation and destruction. Creating and destroying threads consumes system resources, but by reusing threads, a thread pool avoids frequent resource operations, thereby enhancing system performance.
• A thread pool controls and optimizes system resource utilization. By controlling the number of threads, it maximizes machine performance and improves the efficiency of resource usage.
• A thread pool can improve response time. By pre-creating threads and handling tasks concurrently with multi-threading, a thread pool enhances task response speed and system concurrency performance.
The core implementation class of the Java thread pool is ThreadPoolExecutor, and its class inheritance relationship is illustrated in the figure, in which the core method is as follows:
Part of the core method of ThreadPoolExecutor
execute(Runnable r): No return value, just submit a task to the thread pool for processing.
submit (Runnable r): The return value is Future type. When the task is processed and the return value is obtained through the get() method of Future, null is obtained.
submit(Runnable r, Object result): The return value is Future type. When the task is processed and the return value is obtained through the get() method of Future, the result is the second parameter passed in.
shutdown (): Close the thread pool and do not accept new tasks but wait for the tasks in the queue to be processed before they can be truly closed.
shutdownNow (): Immediately close the thread pool, do not accept new tasks, no longer process tasks in the waiting queue, and interrupt the executing threads at the same time.
setCorePoolSize(int corePoolSize): Specify the number of core threads.
setKeepAliveTime(long time, TimeUnit unit): Set the idle time of a thread.
setMaximumPoolSize(int maximumPoolSize): Set the maximum number of threads.
setRejectedExecutionHandler(RejectedExecutionHandler rh): Set the rejection policy.
setThreadFactory(ThreadFactory tf): Set the thread factory.
beforeExecute (Thread t, Runnable r): Hook function before the task is executed. This is an empty function. Users can inherit the ThreadPoolExecutor and rewrite this method to implement the logic.
afterExecute (Runnable r, Throwable t): Hook function after the task is executed. This is an empty function. Users can inherit the ThreadPoolExecutor and rewrite this method to implement the logic.• RUNNING: Once the thread pool is created, it is in the RUNNING state, the number of tasks is 0, and it can receive new tasks and process queued tasks.
• SHUTDOWN: It can not receive new tasks but can process queued tasks. When the shutdown() method of the thread pool is called, the status of the thread pool changes from RUNNING to SHUTDOWN.
• STOP: It can not receive new tasks, process queued tasks, but can interrupt tasks that are being processed. When the shutdownNow() method of the thread pool is called, the status of the thread pool changes from RUNNING or SHUTDOWN to STOP.
• TIDYING: When the thread pool is in the SHUTDOWN state, the task queue is empty and the task in execution is empty. When the thread pool is in the STOP state and the task in execution in the thread pool is empty, the status of the thread pool changes to the TIDYING state and the terminated() method will be executed. This method is an empty implementation in the thread pool and can be overridden to handle it accordingly.
• TERMINATED: The thread pool is terminated completely. After the terminated() method is executed in the TIDYING state, the status of the thread pool changes from TIDYING to TERMINATED.
• Can the core threads of the thread pool be recycled?
The ThreadPoolExecutor does not recycle the core thread by default but provides an allowCoreThreadTimeOut(boolean value) method. When the parameter is true, you can recycle the core thread after reaching the thread idle time. In the business code, if the thread pool is used periodically, you can consider setting this parameter to true.
• Can I create a thread in the thread pool before I submit a task?
ThreadPoolExecutor provides two methods:
prestartCoreThread(): Start a thread and wait for the task. If the number of core threads has been reached, this method returns false, otherwise, it returns true.
prestartAllCoreThreads(): All core threads are started, and the number of successfully started core threads is returned.
With this setting, the creation of the core thread can be completed before the task is submitted, thus realizing the effect of preheating the thread pool.
First, the ctl is obtained. The ctl consists of 32 bits. The upper 3 bits record the status of the thread pool, and the lower 29 bits record the number of working threads in the thread pool. After obtaining the ctl, it determines whether the current number of working threads is less than the number of core threads. If it is less than the number of core threads, it creates a core thread to execute tasks. Otherwise, it attempts to add tasks to create non-core threads.
In the double-layer endless loop, ctl is still obtained to check the status of the current thread pool. After the check is passed, cas will be tried to increase the number of working threads in the inner endless loop. Only if the increase is successful can cas jump out of the outer for loop and actually start to create threads.
To create a thread, you must lock the thread to ensure safe concurrency. The Worker class is used to encapsulate the Thread object in the thread pool. When the thread pool is running or in the shutdown state, you can create a thread and execute tasks in the blocking queue.
After the thread is successfully created and added to the thread pool, the start() method is called to start the thread and execute the task.
In the runWorker() method, the first task is initially the one encapsulated in the Worker object, and then, within a while() loop, tasks are continuously fetched from the blocking queue for execution, achieving thread reuse. One detail to note is that the source code directly throws exceptions from the task.run() method without catching them.
Check the status of the thread pool. If it is not in a running state, and if it is not in shutdown and the blocking queue is not empty, decrement the number of worker threads by 1 and return null. Note that this only decrements the number of worker threads and does not actually destroy the thread; the logic for destroying threads is handled in processWorkerExit().
Provide two blocking methods to obtain the tasks in the blocking queue, depending on whether timeout control is required.
When a thread encounters an exception during execution or fails to retrieve a blocking task, it will enter this method.
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler)1.corePoolSize: the number of core threads
2.maximumPoolSize: the maximum number of threads
3.keepAliveTime: the idle time of the thread
4.unit: unit of idle time (seconds, minutes, hours, etc.)
5.workQueue: waiting queue
6.threadFactory: thread factory
7.handler: reject policy
ThreadPoolExecutor.AbortPolicy: Discard the task and throw the RejectedExecutionException exception.
ThreadPoolExecutor.DiscardPolicy: Also discard the task, but do not throw an exception.
ThreadPoolExecutor.DiscardOldestPolicy: Discard the task at the top of the queue and retry the task (repeat the process).
ThreadPoolExecutor.CallerRunsPolicy: The calling thread directly processes the task (possibly the main thread Main) to ensure that each task is completed.It is recommended to use a custom thread factory that overrides the thread creation method, allowing for customization of thread names, priorities, and other attributes. This facilitates easier troubleshooting.
• How do I select the appropriate thread pool parameters?
CPU-intensive tasks (N +1): These tasks consume CPU resources. You can set the number of threads to N (number of CPU cores) +1. An extra thread beyond the number of CPU cores is used to prevent the impact of occasional page faults or other issues that might cause task interruptions. Once a task is paused, the CPU will be idle. In such cases, the extra thread can make full use of the CPU's idle time.
I/O intensive task (2N): In such tasks, the system spends most of its time handling I/O interactions. During the periods when a thread is handling I/O, it does not occupy the CPU. This allows the CPU to be allocated to other threads during these times. Therefore, for I/O-intensive tasks, you can configure additional threads, with a specific calculation method being 2N.
For example, when a user queries a product detail page, it involves retrieving a series of related information such as price, discounts, inventory, and basic details. From a user experience perspective, it's desirable to minimize the response time of the product detail page. In this case, you can consider using a thread pool to concurrently query information such as price, discounts, and inventory, and then aggregate the results before returning them, thereby reducing the overall API response time. In this case, where the purpose of the thread pool is to achieve the fastest response time, you might consider not setting a queue to buffer concurrent tasks. Instead, configure a larger corePoolSize and maxPoolSize to handle as many tasks concurrently as possible.
For example, when synchronizing product supply with channels in a project, where a large volume of product data needs to be queried and synchronized with the channels, you might consider using a thread pool to efficiently handle the batch tasks. The purpose of this thread pool is to use limited machine resources to process as many tasks as possible per unit time and improve system throughput. Therefore, it is necessary to set blocking queue buffer tasks and adjust the appropriate corePoolSize according to the task scenario.
Use Executors to create a specific thread pool. The thread pool parameters are relatively fixed and are not recommended.
Executors is a utility class in a java.util.concurrent package that makes it easy for us to create several thread pools with specific parameters.
• FixedThreadPool: a thread pool with a fixed number of threads, unbounded blocking queue.
• CachedThreadPool: the number of threads can be dynamically scaled. The maximum number of threads is Integer.MAX_VALUE.
• SingleThreadPool: the thread of a single thread, the number of core threads, and the maximum number of threads are both 1, unbounded blocking queue.
...
It is recommended to use the hungry singleton pattern to create a thread pool object. It supports flexible parameter configuration, creates the thread pool object during class loading, and only instantiates a single object. It then encapsulates a unified method for obtaining the thread pool object, exposing it for use in business code. See the example code below:
public class TestThreadPool {
/**
* Thread pool
*/
private static ExecutorService executor = initDefaultExecutor();
/**
* Unified method for obtaining thread pool objects
*/
public static ExecutorService getExecutor() {
return executor;
}
private static final int DEFAULT_THREAD_SIZE = 16;
private static final int DEFAULT_QUEUE_SIZE = 10240;
private static ExecutorService initDefaultExecutor() {
return new ThreadPoolExecutor(DEFAULT_THREAD_SIZE, DEFAULT_THREAD_SIZE,
300, TimeUnit.SECONDS,
new ArrayBlockingQueue<>(DEFAULT_QUEUE_SIZE),
new DefaultThreadFactory(),
new ThreadPoolExecutor.CallerRunsPolicy());
}
}• Can a thread pool object defined by a local variable be collected as garbage after the method ends?
public static void main(String[] args) {
test1();
test2();
}
public static void test1(){
Object obj = new Object();
System.out.println("Implementation of Method 1 Completed");
}
public static void test2(){
ExecutorService executorService = Executors.newFixedThreadPool(10);
executorService.execute(new Runnable() {
@Override
public void run() {
System.out.println("Implementation of Method 2 Completed");
}
});
}To begin with the conclusion, in the code shown above, obj is a local variable defined within the test1() method. Normally, local variables are stored on the stack. When the method ends, the stack frame is popped, and the local variables within that stack frame are also destroyed. At this point, no variables point to the new Object() instance in the heap, so the heap object created by new Object() can be garbage collected. Similarly, executorService is also a local variable defined within the method, but after the method ends, active threads still exist in the thread pool. According to the GC Roots reachability analysis principle, objects that can serve as GC Roots include:
• Objects referenced in the virtual machine stack (local variable table in the stack frame).
• The object referenced by the class static property in the method area.
• The object referenced by the constant in the method area.
• The object referenced by the native JNI method in the native stack.
• The running threads.
Therefore, after the test2() method completes, the threads in the thread pool will enter a blocked state in getTask(), but they remain active threads. At this point, the thread pool object in the heap is still reachable through GC Roots, so it will not be garbage collected.
To prove the above conclusion, we only need to prove that the running thread object holds a reference to the thread pool object. In source code parsing, we know that the threads in the thread pool are encapsulated by Worker objects, so we only need to find the reference relationship between Worker and ThreadPoolExecutor. In the source code, Worker is an inner class in ThreadPoolExecutor class.
private final class Worker
extends AbstractQueuedSynchronizer
implements RunnableHowever, we do not see that the Worker class directly references the external thread pool object. Could it be that if the thread pool object is unreachable through GC Roots, it can be garbage collected? However, if the thread pool is garbage collected while the threads within it are still alive, this is quite contradictory.
In fact, this issue involves the reference relationship between Java's inner and outer classes. You can refer to the following demo:
public class Outer {
private String name;
private Inner inner;
public int outerMethod() {
return 1;
}
/**
*Non-static inner classes
*/
class Inner {
private void innerMethod() {
//The method of the outer class can be called directly in the non-static inner class.
outerMethod();
}
private String address;
}
}In a non-static inner class, the method of the outer class can be directly called without instantiating the outer class object. The reason is that in Java, the non-static inner class will hold a reference to the outer class. Javac decompilation can be used to verify this conclusion. In the Outer$Inner.class file generated by decompilation, the outer class will be used as the parameter of the non-static inner class construction method. That is, the non-static inner class will hold a reference to the outer class,
class Outer$Inner {
private String address;
Outer$Inner(Outer var1) {
this.this$0 = var1;
}
private void innerMethod() {
this.this$0.outerMethod();
}
}Why emphasize non-static inner classes? Because static inner classes do not hold a reference to the outer class.
public class Outer {
private String name;
private Inner inner;
public int outerMethod() {
return 1;
}
/**
* Static inner classes
*/
static class Inner {
private String address;
}
}The javac decompiled Outer$Inner.class file does not reference the outer class
class Outer$Inner {
private String address;
Outer$Inner() {
}
}This issue leads to two insights:
1) Avoid defining thread pool objects as local variables in your code. This not only leads to the frequent creation of thread pool objects, which goes against the design principle of thread reuse but also potentially causes memory leaks if the local variable's thread pool object cannot be garbage collected in a timely manner.
2) In the business code, the priority is to define static inner classes rather than non-static inner classes, and can effectively prevent memory leakage risk.
public class FartherAndSonTask {
public static ExecutorService executor= TestThreadPool.getExecutor();
public static void main(String[] args) throws Exception {
FatherTask fatherTask = new FatherTask();
Future<String> future = executor.submit(fatherTask);
future.get();
}
/**
* Parent tasks, which asynchronously execute subtasks.
*/
static class FatherTask implements Callable<String> {
@Override
public String call() throws Exception {
System.out.println("Start parent task")
SonTask sonTask = new SonTask();
Future<String> future = executor.submit(sonTask);
String s = future.get();
System.out.println("The parent task has obtained the execution result of the subtask").
return s;
}
}
/**
* Subtasks
*/
static class SonTask implements Callable<String> {
@Override
public String call() throws Exception {
//Process some business logic.
System.out.println("Subtask execution completed").
return null;
}
}
}Interdependent tasks are submitted to the same thread pool. The parent task depends on the execution result of the subtask. When the parent task get() executes the result, the thread may be blocked because the subtask has not been completed. If there are too many submitted tasks, the threads in the thread pool are occupied and blocked by similar parent tasks, resulting in no threads to execute the subtasks in the task queue, resulting in a deadlock of thread starvation.
• Use different thread pools to isolate tasks that are interdependent.
• Call the future.get() method to set the timeout period. This prevents thread blocking but still causes a large number of timeout exceptions.
• execute(Runnable r): No return value, just submit a task to the thread pool for processing, lightweight method, suitable for processing tasks that do not need to return results.
• submit(Runnable r): The return value is Future type. The future can be used to check whether the task has been completed and obtain the result of the task. It is suitable for tasks that need to process the returned result.
For example, the following code is used to push pricing information for calendar rooms to third-party channels. Due to the extensive range of calendar room supply, thread pool technology is employed to improve the efficiency of the pricing push. Additionally, since the entire push task takes a considerable amount of time, and to prevent task interruption, it is necessary to record the execution progress of the push task and implement a "resume from breakpoint" feature. To achieve this, the submit() method is used to submit subtasks, and then the execution results of all subtasks are blocked and retrieved to update the progress of the push task.
private void asyncSupplyPriceSync(List<Long> shidList, SupplyPriceSyncMsg msg) {
if (CollectionUtils.isEmpty(shidList)) {
return;
}
PlatformLogUtil.logInfo("Total number of asynchronous push hotel quotation information supplies:", shidList.size()).
final Map<String, Future<?>> futures = Maps.newLinkedHashMap();
//Batch submission for thread pool processing.
Lists.partition(shidList, SwitchConfig.HOTEL_PRICE_ASYNC_LIST_SIZE)
.forEach(subList -> {
try {
futures.put(UUID.randomUUID().toString(), executorService
.submit(() -> batchSupplyPriceSync(subList, msg)));
} catch (Exception e) {
PlatformLogUtil.logFail("The subtask of asynchronously pushing quotation information to the thread pool is executed abnormally", LogListUtil.newArrayList(subList), e).
}
});
//Blocking. The result will not be returned until all subtasks have been processed.
futures.forEach((uuid, future) -> {
try {
future.get();
} catch (InterruptedException | ExecutionException e) {
PlatformLogUtil.logFail("The execution result of the asynchronous push quotation information acquisition subtask is abnormal", LogListUtil.newArrayList(e)).
}
});
}public class ExceptionTest {
public static ExecutorService executor = TestThreadPool.getExecutor();
public static void main(String[] args) {
executor.execute(() -> test("normal")).
executor.execute(() -> test("normal")).
executor.execute(() -> test("Task execution exception")).
executor.execute(() -> test("normal")).
executor.shutdown();
}
public static void test(String str) {
String result = "The current ThreadName is" + Thread.currentThread().getName() + ": Result" + str.
if (str.equals("Task execution exception")) {
throw new RuntimeException(result + "**** Execution exception").
} else {
System.out.println(result);
}
}
}• If the thread executing the task in the thread pool is abnormal, will the abnormal thread be destroyed? Can other tasks be performed normally?
The preceding code execution result is shown in the following figure:
What can be found is that:
1) The thread that executes the task in the thread pool is abnormal but does not affect the execution of other tasks. In addition, execute() submits the task and directly prints the exception information. If you use submit() to submit a task, what will the console output? Students who are interested in this can explore it.
2) Note that the printed thread names are 1,2,5,3. Why is there no printed thread name 4?
First of all, we can confirm that a new thread 5 was created after the thread exception. As for how thread 3 is handled and why thread 4 is not printed, we can analyze it from the source code. After executing the task.run() method, if there is an exception, the runWorker() method will throw an exception and enter three finally code blocks in turn.
In the processWorkerExit(w, completedAbruptly) method, you can see that if the running thread pool has a thread execution exception, it will call workers.remove() to remove the current thread and call addWorker() to recreate a new thread.
Therefore, the two actions of destroying the thread and recreating the thread in task 3 and creating the thread in task 4 have timing issues. For more information, see the following figure:
So where does the abnormal information that controls printing come from?
After runWorker() is executed, since the exception executed by task.run() is not caught, the jvm calls back java.lang.Thread#dispatchUncaughtException method will eventually process the uncaught exception information in the thread pool and finally call the java.lang.ThreadGroup#uncaughtException. As you can see, we are supported here to customize the uncaught exception handler UncaughtExceptionHandler, otherwise, the exception information will be printed directly by default.
Therefore, in the business code, please catch exceptions in subtasks. Otherwise, worker threads in the thread pool will be frequently destroyed and created, resulting in a waste of resources and violating the design principle of thread reuse.
Disclaimer: The views expressed herein are for reference only and don't necessarily represent the official views of Alibaba Cloud.
Embrace Open Source Ecosystem with Alibaba Load Balancer Controller V1.2.0
1,037 posts | 255 followers
FollowAlibaba Cloud Community - July 11, 2023
block - September 14, 2021
Alibaba Cloud Storage - June 19, 2019
Alibaba Clouder - April 13, 2020
Alibaba Cloud Community - May 7, 2024
Alibaba Cloud Native Community - August 23, 2023
1,037 posts | 255 followers
Follow
Best Practices
Follow our step-by-step best practices guides to build your own business case.
Learn More
Web Hosting Solution
Explore Web Hosting solutions that can power your personal website or empower your online business.
Learn More
Web Hosting
Explore how our Web Hosting solutions help small and medium sized companies power their websites and online businesses.
Learn More
EMAS Superapp
Build superapps and corresponding ecosystems on a full-stack platform
Learn MoreMore Posts by Alibaba Cloud Community