David - Musings of an SRE

Ruby Multithreading with Threadwait

Multithreading is nothing new in Ruby but its only in recent times that I’ve found a use for it.

Unlike background processes which runs your task on its own instance independently from the main rails app, Multithreading also runs each threads separately but needs to join back with the main thread in order to continue.

An example use:

@groups.each do |t|
  threads << Thread.new do
    t.process_something
  end
end

# Group
def process_something
  # calls external resources and wait for a reply.
end

In the old days, if I were to not use multithreading it will be just:

@groups.each do |t|
  t.process_something
end

And if one of the group’s “process_something” has a blocking IO, the other groups down the iterative line may never get called.

But something’s missing. You said that all threads have to rejoin their main thread but the first example doesn’t show and rejoin. How do I do that?

# Without Threadwait
@groups.each do |t|
..
end

threads.each{|t| t.join}
# use ThreadWait

require 'thread'
require 'thwait'

@group.each do |t|
..
end

ThreadsWait.all_waits(*threads)

Why Threadwait? Threadwait waits for every thread specified to complete before continuing down the main thread line.

Without threadwait, what happens is that when the threads gets forked, asynchronously your main thread (the one doing the forking) reaches its end of file and if the forked threads haven’t been completed since, it will just kill itself.

Whereas with ThreadWait, it monitors all the threads and only continue down the file after all Threads have been completed and joined. This ensures that your forked thread always runs into completion.

2021 Update

  • Both threads.each{|t| t.join} and ThreadsWait.all_waits(*threads) will both wait till all the threads is completed.
  • ThreadsWait.all_waits can take in a block and will perform the block when a thread completes.

Reference: