cl-threadpool

A thread pool implemented in Common Lisp. Thread pools consist of a bunch of worker threads and a job queue.

The source code is available here.

Examples

Synchronously execute a batch of jobs

(defpackage :cl-threadpool-example-1
  (:documentation "Synchronously execute a batch of jobs")
  (:use :cl))
(in-package :cl-threadpool-example-1)

(defun example()
  (let ((threadpool (cl-threadpool:make-threadpool 5 :name "Example thread pool")))
    (let ((results
           (cl-threadpool:run-jobs
            threadpool
            (list
             (lambda() (sleep 5) "Batch-Job 1")
             (lambda() (sleep 2) "Batch-Job 2")
             (lambda() (sleep 1) "Batch-Job 3")))))
      (format t "~%~a" (first results)) ;; => "Batch-Job 1"
      (format t "~%~a" (second results)) ;; => "Batch-Job 2"
      (format t "~%~a" (third results))) ;; => "Batch-Job 3"
      (cl-threadpool:stop threadpool)))

;; (example)

Add some jobs and retrieve their results in a later point of time

(defpackage :cl-threadpool-example-2
  (:documentation "Add some jobs and retrieve their results in a later point of time")
  (:use :cl))
(in-package :cl-threadpool-example-2)

(defun example()
  (let ((threadpool (cl-threadpool:make-threadpool 5 :name "Example thread pool")))
    (let ((job-1 (cl-threadpool:add-job
          threadpool
          (lambda()
            (sleep 5)
            "Job 1")))
      (job-2 (cl-threadpool:add-job
          threadpool
          (lambda()
            (sleep 5)
            "Job 2"))))
      ;;
      ;; Print status of jobs
      ;;
      (format t "~%job-1 done: ~a" (cl-threadpool:job-done-p job-1)) ;; => "job-1 done: NIL"
      (format t "~%job-2 done: ~a" (cl-threadpool:job-done-p job-2)) ;; => "job-2 done: NIL"
      ;;
      ;; Retrieve job results
      ;;
      (format t "~%~a" (cl-threadpool:job-result job-1)) ;; => "Job 1"
      (format t "~%~a" (cl-threadpool:job-result job-2)) ;; => "Job 2"
      ;;
      ;; Print status of jobs
      ;;
      (format t "~%job-1 done: ~a" (cl-threadpool:job-done-p job-1)) ;; => "job-1 done: T"
      (format t "~%job-2 done: ~a" (cl-threadpool:job-done-p job-2)) ;; => "job-2 done: T"
      (cl-threadpool:stop threadpool))))

;; (example)

Change-Log

Version 1.0.0

Initial release of cl-threadpool.

Version 2.0.0

Version 2 is a major rework of the thread pool with bugfixes, new features and removal of features that have been identified as not being useful.

Breaking changes

New features

Version 3.0.0

This version is the current quicklisp release.

This version introduces futures and shall also be the last version coming with breaking changes.

Breaking changes

New features

Installation

Installation via Quicklisp

(ql:quickload "cl-threadpool")

API

make-threadpool (size &key (name nil))

Instantiates a thread pool. The function has the following arguments:

Returns a thread pool.

run-jobs (pool jobs)

Executes a batch of jobs and returns their results. Blocks until all jobs are done. The function has the following arguments:

May signal one of the following conditions: Returns an ordered list of job results.

add-job (pool job)

Adds a job to the pool. The function has the following arguments:

Returns a future.

stop (pool &key (timeout-seconds nil))

Stops all worker threads. The function returns when all worker threads are no longer alive or when the timeout has been reached or when the pool is stopping or is already stopped. All pending jobs that are not currently being executed by a worker thread will be cancelled by one of the worker threads.

The function does not destroy threads but signals to the worker threads that they are supposed to end. If a worker thread refuses to end it will be left running.

See also pool-stopped-p to check if the pool has successfully been stopped.

threadpoolp (obj)

Returns t if the given object represents a thread pool.

queue-size (pool)

Returns the current length of the job queue.

pool-name (pool)

Returns the name of the pool.

pool-stopped-p (pool)

Returns t if the job queue of the pool is empty and all worker threads have ended.

worker-thread-p (pool)

Returns true if the current thread is a worker thread of the given pool.

job-result (future)

Get the result of a job. If the result is already available it will immediately be returned. Otherwise the function blocks on the completion of the job. The function may signal one of the following conditions:

job-done-p (future)

Returns t if the job is done. A job is done when it has succesfully completed, signalled a condition or has been cancelled.

cancel-job (future)

Cancels a job. The function does nothing when the job is already done.

job-cancelled-p (future)

Returns t if the job has been cancelled.

job-cancellation-error

This condition is signalled when the result of a job is requested but the job has been cancelled.

job-execution-error

This condition is signalled when the result of a job is requested but the job has signalled a condition during its execution.

job-execution-error-pool-name (job-execution-error)

Returns the name of the thread pool that has signalled the given job execution error.

job-execution-error-message (job-execution-error)

Returns the error message of the given job execution error.

*logger*

The logger of cl-threadpool. The default implementation is empty. Refer to test/util/logger.lisp for an implementation using the "verbose" library.

Run tests

(asdf:test-system :cl-threadpool)

Tested Lisp implementations and operating systems

cl-threadpool has been tested with:

Generate documentation

(asdf:load-system :cl-threadpool/doc)
(cl-threadpool-make-doc::make-doc)