Rework parallelism in directory iterator.

Previously, ignore::WalkParallel would invoke the callback for all
*explicitly* given file paths in a single thread, which effectively
meant that `rg pattern foo bar baz ...` didn't actually search foo, bar
and baz in parallel.

The code was structured that way to avoid spinning up workers if no
directory paths were given. The original intention was probably to have
a separate pool of threads responsible for searching, but ripgrep ended
up just reusing the ignore::WalkParallel workers themselves for searching,
and thereby subjected to its sub-par performance in this case.

The code has been restructured so that file paths are sent to the workers,
which brings back parallelism.

Fixes #226
This commit is contained in:
Andrew Gallant
2016-11-09 17:19:40 -05:00
parent 2dce0dc0df
commit 5b73dcc8ab
2 changed files with 36 additions and 28 deletions

View File

@@ -1,3 +1,4 @@
use std::cmp;
use std::env;
use std::io;
use std::path::{Path, PathBuf};
@@ -376,7 +377,7 @@ impl RawArgs {
};
let threads =
if self.flag_threads == 0 {
num_cpus::get()
cmp::min(12, num_cpus::get())
} else {
self.flag_threads
};