Fix Gem executable

- Implement prefix caching of extended mode
- Improved ranking algorithm for extended mode
- Fix nfc conversion bug

Gemfile

@@ -1,4 +0,0 @@
-source 'https://rubygems.org'
-
-# Specify your gem's dependencies in fzf.gemspec
-gemspec

README.md

@@ -16,7 +16,8 @@ fzf requires Ruby (>= 1.8.5).
 Installation
 ------------
 
-Download fzf executable and put it somewhere in your search $PATH.
+Download [fzf executable](https://raw.github.com/junegunn/fzf/master/fzf) and
+put it somewhere in your search $PATH.
 
 ```sh
 mkdir -p ~/bin
@@ -38,15 +39,21 @@ Make sure that ~/bin is included in $PATH.
 export PATH=$PATH:~/bin
 ```
 
-Install as Vim plugin
----------------------
+### Install as Ruby gem
 
-fzf was not designed to be a Vim plugin, but you can use it as one. The only
-reason one might consider using fzf in Vim is its speed. For a very large list
-of files, fzf is significantly faster than native Vim plugins.
+fzf can be installed as a Ruby gem
 
-You can use any Vim plugin manager to install fzf as a Vim plugin. If you don't
-use one, I recommend you try [vim-plug](https://github.com/junegunn/vim-plug).
+```
+gem install fzf
+```
+
+It's a bit easier to install and update the script but the Ruby gem version
+takes slightly longer to start.
+
+### Install as Vim plugin
+
+You can use any Vim plugin manager to install fzf for Vim. If you don't use one,
+I recommend you try [vim-plug](https://github.com/junegunn/vim-plug).
 
 1. [Install vim-plug](https://github.com/junegunn/vim-plug#usage)
 2. Edit your .vimrc
@@ -64,9 +71,12 @@ Usage
 ```
 usage: fzf [options]
 
-  -s, --sort=MAX   Maximum number of matched items to sort. Default: 500
-  +s, --no-sort    Keep the sequence unchanged.
+  -m, --multi      Enable multi-select
+  -x, --extended   Extended-search mode
+  -s, --sort=MAX   Maximum number of matched items to sort. Default: 1000
+  +s, --no-sort    Do not sort the result. Keep the sequence unchanged.
   +i               Case-sensitive match
+  +c, --no-color   Disable colors
 ```
 
 fzf will launch curses-based finder, read the list from STDIN, and write the
@@ -77,10 +87,11 @@ find * -type f | fzf > selected
 ```
 
 Without STDIN pipe, fzf will use find command to fetch the list of
-files (excluding hidden ones).
+files excluding hidden ones. (You can override the default command with
+`FZF_DEFAULT_COMMAND`)
 
 ```sh
-vim `fzf`
+vim $(fzf)
 ```
 
 If you want to preserve the exact sequence of the input, provide `--no-sort` (or
@@ -100,6 +111,26 @@ The following readline key bindings should also work as expected.
 - CTRL-A / CTRL-E
 - CTRL-B / CTRL-F
 - CTRL-W / CTRL-U
+- ALT-B / ALT-F
+
+If you enable multi-select mode with `-m` option, you can select multiple items
+with TAB or Shift-TAB key.
+
+### Extended-search mode
+
+With `-x` or `--extended` option, fzf will start in "extended-search mode".
+
+In this mode, you can specify multiple patterns delimited by spaces,
+such as: `^music .mp3$ sbtrkt !rmx`
+
+| Token    | Description                      | Match type           |
+| -------- | -------------------------------- | -------------------- |
+| `^music` | Items that start with `music`    | prefix-exact-match   |
+| `.mp3$`  | Items that end with `.mp3`       | suffix-exact-match   |
+| `sbtrkt` | Items that match `sbtrkt`        | fuzzy-match          |
+| `!rmx`   | Items that do not match `rmx`    | inverse-fuzzy-match  |
+| `'wild`  | Items that include `wild`        | exact-match (quoted) |
+| `!'fire` | Items that do not include `fire` | inverse-exact-match  |
 
 Usage as Vim plugin
 -------------------
@@ -117,23 +148,27 @@ You can override the command which produces input to fzf.
 let g:fzf_command = 'find . -type f'
 ```
 
+Most of the time, you will prefer native Vim plugins with better integration
+with Vim. The only reason one might consider using fzf in Vim is its speed. For
+a very large list of files, fzf is significantly faster and it does not block.
+
 Useful bash examples
 --------------------
 
 ```sh
 # vimf - Open selected file in Vim
 vimf() {
-  FILE=`fzf` && vim "$FILE"
+  FILE=$(fzf) && vim "$FILE"
 }
 
 # fd - cd to selected directory
 fd() {
-  DIR=`find ${1:-*} -path '*/\.*' -prune -o -type d -print 2> /dev/null | fzf` && cd "$DIR"
+  DIR=$(find ${1:-*} -path '*/\.*' -prune -o -type d -print 2> /dev/null | fzf) && cd "$DIR"
 }
 
 # fda - including hidden directories
 fda() {
-  DIR=`find ${1:-*} -type d 2> /dev/null | fzf` && cd "$DIR"
+  DIR=$(find ${1:-*} -type d 2> /dev/null | fzf) && cd "$DIR"
 }
 
 # fh - repeat history
@@ -143,14 +178,84 @@ fh() {
 
 # fkill - kill process
 fkill() {
-  ps -ef | sed 1d | fzf | awk '{print $2}' | xargs kill -${1:-9}
+  ps -ef | sed 1d | fzf -m | awk '{print $2}' | xargs kill -${1:-9}
 }
 
-# CTRL-T - Open fuzzy finder and paste the selected item to the command line
+# (Assuming you don't use the default CTRL-T and CTRL-R)
+
+# CTRL-T - Paste the selected file path into the command line
 bind '"\er": redraw-current-line'
 bind '"\C-t": " \C-u \C-a\C-k$(fzf)\e\C-e\C-y\C-a\C-y\ey\C-h\C-e\er"'
+
+# CTRL-R - Paste the selected command from history into the command line
+bind '"\C-r": " \C-e\C-u$(history | fzf +s | sed \"s/ *[0-9]* *//\")\e\C-e\er"'
 ```
 
+zsh widgets
+-----------
+
+```sh
+# CTRL-T - Paste the selected file(s) path into the command line
+fzf-file-widget() {
+  local FILES
+  local IFS="
+"
+  FILES=($(
+    find * -path '*/\.*' -prune \
+    -o -type f -print \
+    -o -type l -print 2> /dev/null | fzf -m))
+  unset IFS
+  FILES=$FILES:q
+  LBUFFER="${LBUFFER%% #} $FILES"
+  zle redisplay
+}
+zle     -N   fzf-file-widget
+bindkey '^T' fzf-file-widget
+
+# ALT-C - cd into the selected directory
+fzf-cd-widget() {
+  cd "${$(find * -path '*/\.*' -prune \
+          -o -type d -print 2> /dev/null | fzf):-.}"
+  zle reset-prompt
+}
+zle     -N    fzf-cd-widget
+bindkey '\ec' fzf-cd-widget
+
+# CTRL-R - Paste the selected command from history into the command line
+fzf-history-widget() {
+  LBUFFER=$(history | fzf +s | sed "s/ *[0-9]* *//")
+  zle redisplay
+}
+zle     -N   fzf-history-widget
+bindkey '^R' fzf-history-widget
+```
+
+Tips
+----
+
+### Faster startup with `--disable-gems` options
+
+If you're running Ruby 1.9 or above, you can improve the startup time with
+`--disable-gems` option to Ruby.
+
+- `time ruby ~/bin/fzf -h`
+    - 0.077 sec
+- `time ruby --disable-gems ~/bin/fzf -h`
+    - 0.025 sec
+
+Define fzf alias with the option as follows:
+
+```sh
+alias fzf='ruby --disable-gems ~/bin/fzf'
+```
+
+### Incorrect display on Ruby 1.8
+
+It is reported that the output of fzf can become unreadable on some terminals
+when it's running on Ruby 1.8. If you experience the problem, upgrade your Ruby
+to 1.9 or above. Ruby 1.9 or above is also required for displaying Unicode
+characters.
+
 License
 -------
 

Rakefile

@@ -1 +1,8 @@
 require "bundler/gem_tasks"
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |test|
+  test.pattern = 'test/**/test_*.rb'
+  test.verbose = true
+end
+

fzf

@@ -10,7 +10,7 @@
 # URL:         https://github.com/junegunn/fzf
 # Author:      Junegunn Choi
 # License:     MIT
-# Last update: October 29, 2013
+# Last update: November 17, 2013
 #
 # Copyright (c) 2013 Junegunn Choi
 #
@@ -35,22 +35,46 @@
 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
-def usage x
-  puts %[usage: fzf [options]
+require 'thread'
+require 'curses'
+require 'set'
 
-  -s, --sort=MAX   Maximum number of matched items to sort. Default: 500.
-  +s, --no-sort    Do not sort the result. Keep the sequence unchanged.
-  +i               Case-sensitive match]
-  exit x
+class FZF
+  C = Curses
+  attr_reader :rxflag, :sort, :color, :multi
+
+  class AtomicVar
+    def initialize value
+      @value = value
+      @mutex = Mutex.new
     end
 
-stdout = $stdout.clone
-$stdout.reopen($stderr)
+    def get
+      @mutex.synchronize { @value }
+    end
 
-usage 0 unless (%w[--help -h] & ARGV).empty?
-@rxflag = ARGV.delete('+i') ? 0 : Regexp::IGNORECASE
-@sort   = (ARGV.delete('+s') || ARGV.delete('--no-sort')) ? nil : 500
-rest    = ARGV.join ' '
+    def set value = nil
+      if block_given?
+        @mutex.synchronize { @value = yield @value }
+      else
+        @mutex.synchronize { @value = value }
+      end
+    end
+
+    def method_missing sym, *args, &blk
+      @mutex.synchronize { @value.send(sym, *args, &blk) }
+    end
+  end
+
+  def initialize argv, source = $stdin
+    usage 0 unless (%w[--help -h] & argv).empty?
+    @rxflag = argv.delete('+i') ? 0 : Regexp::IGNORECASE
+    @sort   = %w[+s --no-sort].map   { |e| argv.delete e }.compact.empty? ?
+      ENV.fetch('FZF_DEFAULT_SORT', 1000).to_i : nil
+    @color  = %w[+c --no-color].map  { |e| argv.delete e }.compact.empty?
+    @multi  = !%w[-m --multi].map    { |e| argv.delete e }.compact.empty?
+    @xmode  = !%w[-x --extended].map { |e| argv.delete e }.compact.empty?
+    rest    = argv.join ' '
     if sort = rest.match(/(-s|--sort=?) ?([0-9]+)/)
       usage 1 unless @sort
       @sort = sort[2].to_i
@@ -58,20 +82,46 @@ if sort = rest.match(/(-s|--sort=?) ?([0-9]+)/)
     end
     usage 1 unless rest.empty?
 
-require 'thread'
-require 'curses'
-
+    @source   = source
     @mtx      = Mutex.new
-@smtx     = Mutex.new
     @cv       = ConditionVariable.new
-@lists    = []
-@new      = []
-@query    = ''
-@matches  = []
-@count    = 0
-@cursor_x = 0
-@vcursor  = 0
     @events   = {}
+    @new      = []
+    @queue    = Queue.new
+    @cursor_x = AtomicVar.new(0)
+    @query    = AtomicVar.new('')
+    @matches  = AtomicVar.new([])
+    @count    = AtomicVar.new(0)
+    @vcursor  = AtomicVar.new(0)
+    @vcursors = AtomicVar.new(Set.new)
+    @spinner  = AtomicVar.new('-\|/-\|/'.split(//))
+    @selects  = AtomicVar.new({}) # ordered >= 1.9
+    @main     = Thread.current
+    @stdout   = $stdout.clone
+    @plcount  = 0
+  end
+
+  def start
+    $stdout.reopen($stderr)
+
+    init_screen
+    start_reader
+    start_renderer
+    start_search
+    start_loop
+  end
+
+  def usage x
+    $stderr.puts %[usage: fzf [options]
+
+    -m, --multi      Enable multi-select
+    -x, --extended   Extended-search mode
+    -s, --sort=MAX   Maximum number of matched items to sort. Default: 1000
+    +s, --no-sort    Do not sort the result. Keep the sequence unchanged.
+    +i               Case-sensitive match
+    +c, --no-color   Disable colors]
+    exit x
+  end
 
   case RUBY_PLATFORM
   when /darwin/
@@ -104,12 +154,33 @@ when /darwin/
         ret
       end
 
-    def self.nfc str, offset
+      def self.to_nfc arr
+        [NFC_BEGIN + arr[0] * JJCOUNT +
+         (arr[1] || 0) * JONGSUNGS +
+         (arr[2] || 0)].pack('U*')
+      end
+
+      if String.method_defined?(:each_char)
+        def self.split str
+          str.each_char.to_a
+        end
+      else
+        def self.split str
+          str.split('')
+        end
+      end
+
+      def self.nfc str, offsets = []
         ret  = ''
         omap = []
         pend = []
-      str.split(//).each_with_index do |c, idx|
-        cp = c.ord
+        split(str).each_with_index do |c, idx|
+          cp =
+            begin
+              c.ord
+            rescue Exception
+              next
+            end
           omap << ret.length
           unless pend.empty?
             if cp >= JUNGSUNG && cp < JUNGSUNG + JUNGSUNGS
@@ -120,9 +191,7 @@ when /darwin/
               next
             else
               omap[-1] = omap[-1] + 1
-            ret << [NFC_BEGIN + pend[0] * JJCOUNT +
-                                (pend[1] || 0) * JONGSUNGS +
-                                (pend[2] || 0)].pack('U*')
+              ret << to_nfc(pend)
               pend.clear
             end
           end
@@ -132,24 +201,32 @@ when /darwin/
             ret << c
           end
         end
-      return [ret, offset.map { |o| omap[o] || (omap.last + 1) }]
+        ret << to_nfc(pend) unless pend.empty?
+        return [ret,
+                offsets.map { |pair|
+                  b, e = pair
+                  [omap[b] || 0, omap[e] || ((omap.last || 0) + 1)] }]
       end
     end
 
-  def convert_query q
-    UConv.nfd(q).split(//)
-  end
-
     def convert_item item
       UConv.nfc(*item)
     end
+
+    class Matcher
+      def convert_query q
+        UConv.nfd(q).split(//)
+      end
+    end
   else
+    def convert_item item
+      item
+    end
+
+    class Matcher
       def convert_query q
         q.split(//)
       end
-
-  def convert_item item
-    item
     end
   end
 
@@ -160,45 +237,47 @@ def emit event
     end
   end
 
-C = Curses
   def max_items; C.lines - 2; end
   def cursor_y;  C.lines - 1; end
   def cprint str, col
     C.attron(col) do
-    C.addstr str
+      addstr_safe str
     end if str
   end
+  def addstr_safe str
+    C.addstr str.gsub("\0", '')
+  end
 
   def print_input
     C.setpos cursor_y, 0
     C.clrtoeol
-  cprint '> ', color(:blue, true)
+    cprint '> ', color(:prompt, true)
     C.attron(C::A_BOLD) do
-    C.addstr @query
+      C.addstr @query.get
     end
   end
 
-def print_info progress = true, msg = nil
-  @fan ||= '-\|/-\|/'.split(//)
+  def print_info msg = nil
     C.setpos cursor_y - 1, 0
     C.clrtoeol
     prefix =
-    if fan = @fan.shift
-      @fan.push fan
-      cprint fan, color(:fan, true)
+      if spinner = @spinner.first
+        cprint spinner, color(:spinner, true)
         ' '
       else
         '  '
       end
     C.attron color(:info, false) do
-    progress &&= "#{prefix}#{@matches.length}/#{@count}"
-    C.addstr progress if progress
+      C.addstr "#{prefix}#{@matches.length}/#{@count.get}"
+      if (selected = @selects.length) > 0
+        C.addstr " (#{selected})"
+      end
       C.addstr msg if msg
     end
   end
 
   def refresh
-  C.setpos cursor_y, 2 + ulen(@query[0, @cursor_x])
+    C.setpos cursor_y, 2 + width(@query[0, @cursor_x.get])
     C.refresh
   end
 
@@ -206,29 +285,128 @@ def ctrl char
     char.to_s.ord - 'a'.ord + 1
   end
 
+  def format line, limit, offsets
+    offsets ||= []
+    maxe = offsets.map { |e| e.last }.max || 0
+
+    # Overflow
+    if width(line) > limit
+      ewidth = width(line[0...maxe])
+      # Stri..
+      if ewidth <= limit - 2
+        line, _ = trim line, limit - 2, false
+        line << '..'
+      # ..ring
+      else
+        # ..ri..
+        line = line[0...maxe] + '..' if ewidth < width(line) - 2
+        line, diff = trim line, limit - 2, true
+        offsets = offsets.map { |pair|
+          b, e = pair
+          b += 2 - diff
+          e += 2 - diff
+          b = [2, b].max
+          [b, e]
+        }
+        line = '..' + line
+      end
+    end
+
+    tokens = []
+    index = 0
+    offsets.select  { |pair| pair.first < pair.last }.
+            sort_by { |pair| pair }.each do |pair|
+      b, e = pair.map { |x| [index, x].max }
+      tokens << [line[index...b], false]
+      tokens << [line[b...e], true]
+      index = e
+    end
+    tokens << [line[index..-1], false]
+    tokens.reject { |pair| pair.first.empty? }
+  end
+
+  def print_item row, tokens, chosen, selected
+    # Cursor
+    C.setpos row, 0
+    C.clrtoeol
+    cprint chosen ? '>' : ' ', color(:cursor, true)
+    cprint selected ? '>' : ' ',
+      chosen ? color(:chosen) : (selected ? color(:selected, true) : 0)
+
+    # Highlighted item
+    C.attron color(:chosen, true) if chosen
+    tokens.each do |pair|
+      token, highlighted = pair
+
+      if highlighted
+        cprint   token, color(chosen ? :match! : :match, chosen)
+        C.attron color(:chosen, true) if chosen
+      else
+        addstr_safe token
+      end
+    end
+    C.attroff color(:chosen, true) if chosen
+  end
+
+  def sort_by_rank list
+    list.sort_by { |tuple|
+      line, offsets = tuple
+      matchlen = 0
+      pe = nil
+      offsets.sort.each do |pair|
+        b, e = pair
+        b    = pe if pe && pe > b
+        pe   = e
+        matchlen += e - b
+      end
+      [matchlen, line.length, line]
+    }
+  end
+
   if RUBY_VERSION.split('.').map { |e| e.rjust(3, '0') }.join > '001009'
-  def ulen str
-    @urx ||= Regexp.new '\p{Han}|\p{Katakana}|\p{Hiragana}|\p{Hangul}'
-    str.gsub(@urx, '  ').length
+    @@wrx = Regexp.new '\p{Han}|\p{Katakana}|\p{Hiragana}|\p{Hangul}'
+    def width str
+      str.gsub(@@wrx, '  ').length
+    end
+
+    def trim str, len, left
+      width = width str
+      diff  = 0
+      while width > len
+        width -= (left ? str[0, 1] : str[-1, 1]) =~ @@wrx ? 2 : 1
+        str    = left ? str[1..-1] : str[0...-1]
+        diff  += 1
+      end
+      [str, diff]
     end
   else
-  def ulen str
+    def width str
       str.length
     end
 
-  class String
+    def trim str, len, left
+      diff = str.length - len
+      if diff > 0
+        [left ? str[diff..-1] : str[0...-diff], diff]
+      else
+        [str, 0]
+      end
+    end
+
+    class ::String
       def ord
         self.unpack('c').first
       end
     end
 
-  class Fixnum
+    class ::Fixnum
       def ord
         self
       end
     end
   end
 
+  def init_screen
     C.init_screen
     C.start_color
     dbg =
@@ -240,16 +418,17 @@ dbg =
       end
     C.raw
     C.noecho
+
+    if @color
       if C.can_change_color?
-  fg = ENV.fetch('FZF_FG', 252).to_i
-  bg = ENV.fetch('FZF_BG', 236).to_i
         C.init_pair 1, 110, dbg
         C.init_pair 2, 108, dbg
-  C.init_pair 3, fg + 2, bg
-  C.init_pair 4, 151,    bg
+        C.init_pair 3, 254, 236
+        C.init_pair 4, 151, 236
         C.init_pair 5, 148, dbg
         C.init_pair 6, 144, dbg
-  C.init_pair 7, 161,    bg
+        C.init_pair 7, 161, 236
+        C.init_pair 8, 168, 236
       else
         C.init_pair 1, C::COLOR_BLUE,    dbg
         C.init_pair 2, C::COLOR_GREEN,   dbg
@@ -258,52 +437,67 @@ else
         C.init_pair 5, C::COLOR_GREEN,   dbg
         C.init_pair 6, C::COLOR_WHITE,   dbg
         C.init_pair 7, C::COLOR_RED,     C::COLOR_BLACK
+        C.init_pair 8, C::COLOR_MAGENTA, C::COLOR_BLACK
       end
 
-def color sym, bold = false
-  C.color_pair([:blue, :match, :chosen,
-                :match!, :fan, :info, :red].index(sym) + 1) |
+      def self.color sym, bold = false
+        C.color_pair([:prompt, :match, :chosen, :match!,
+                      :spinner, :info, :cursor, :selected].index(sym) + 1) |
           (bold ? C::A_BOLD : 0)
       end
+    else
+      def self.color sym, bold = false
+        case sym
+        when :chosen
+          bold ? C::A_REVERSE : 0
+        when :match
+          C::A_UNDERLINE
+        when :match!
+          C::A_REVERSE | C::A_UNDERLINE
+        else
+          0
+        end | (bold ? C::A_BOLD : 0)
+      end
+    end
+  end
 
-@read =
-  if $stdin.tty?
-    if !`which find`.empty?
+  def start_reader
+    stream =
+      if @source.tty?
+        if default_command = ENV['FZF_DEFAULT_COMMAND']
+          IO.popen(default_command)
+        elsif !`which find`.empty?
           IO.popen("find * -path '*/\\.*' -prune -o -type f -print -o -type l -print 2> /dev/null")
         else
           exit 1
         end
       else
-    $stdin
+        @source
       end
 
-reader = Thread.new {
-  while line = @read.gets
+    Thread.new do
+      while line = stream.gets
         emit(:new) { @new << line.chomp }
       end
       emit(:loaded) { true }
-  @smtx.synchronize { @fan = [] }
-}
+      @spinner.clear
+    end
+  end
 
-main     = Thread.current
+  def start_search
+    matcher  = (@xmode ? ExtendedFuzzyMatcher : FuzzyMatcher).new @rxflag
     searcher = Thread.new {
+      lists   = []
       events  = {}
       fcache  = {}
-  matches = []
-  mcount  = 0      # match count
-  plcount = 0      # prev list count
       q       = ''
-  vcursor = 0
-  zz      = [0, 0]
-  started = false
+      delay   = -5
 
       begin
         while true
-      wait_for_completion = nil
           @mtx.synchronize do
             while true
               events.merge! @events
-          wait_for_completion = !@sort && !events[:loaded]
 
               if @events.empty? # No new events
                 @cv.wait @mtx
@@ -313,215 +507,206 @@ searcher = Thread.new {
               break
             end
 
-        if !wait_for_completion && events[:new]
-          @lists << [@new, {}]
-          @count += @new.length
+            if events[:new]
+              lists << @new
+              @count.set { |c| c + @new.length }
+              @spinner.set { |spinner|
+                if e = spinner.shift
+                  spinner.push e
+                end; spinner
+              }
               @new = []
-          fcache  = {}
+              fcache.clear
             end
           end#mtx
 
-      if wait_for_completion
-        @smtx.synchronize do
-          print_info false, " +#{@new.length}"
-          print_input
-          refresh
-          sleep 0.1
-        end
-        next
-      end
-
-      new_search = events[:key] || events[:new]
-      user_input = events[:key] || events[:vcursor]
-
-      if new_search && !@lists.empty?
-        events.delete :new
-        q = events.delete(:key) || q
-        regexp = q.empty? ? nil :
-          Regexp.new(convert_query(q).inject('') { |sum, e|
-            e = Regexp.escape e
-            sum << "#{e}[^#{e}]*?"
-          }, @rxflag)
+          new_search = events[:key] || events.delete(:new)
+          user_input = events[:key]
+          progress   = 0
+          started_at = Time.now
 
+          if new_search && !lists.empty?
+            q, cx = events.delete(:key) || [q, 0]
+            empty = matcher.empty?(q)
             matches = fcache[q] ||=
               begin
-            @smtx.synchronize do
-              print_info true, ' ..'
-              print_input
-              refresh
-            end unless q.empty?
-
                 found = []
                 skip  = false
-            @lists.each do |pair|
-              @mtx.synchronize { skip = @events[:key] }
+                cnt   = 0
+                lists.each do |list|
+                  cnt += list.length
+                  skip = @mtx.synchronize { @events[:key] }
                   break if skip
 
-              list, cache = pair
-              found.concat(cache[q] ||= begin
-                prefix, suffix = @query[0, @cursor_x], @query[@cursor_x..-1] || ''
-                prefix_cache = suffix_cache = nil
-
-                (prefix.length - 1).downto(1) do |len|
-                  break if prefix_cache = cache[prefix[0, len]]
+                  if !empty && (progress = 100 * cnt / @count.get) < 100 && Time.now - started_at > 0.5
+                    render { print_info " (#{progress}%)" }
                   end
 
-                0.upto(suffix.length - 1) do |idx|
-                  break if suffix_cache = cache[suffix[idx..-1]]
-                end unless suffix.empty?
-
-                partial_cache = [prefix_cache, suffix_cache].compact.sort_by { |e| e.length }.first
-                (partial_cache ? partial_cache.map { |e| e.first } : list).map { |line|
-                  if regexp
-                    # Ignore errors: e.g. invalid byte sequence in UTF-8
-                    md = line.match(regexp) rescue nil
-                    md ? [line, md.offset(0)] : nil
-                  else
-                    [line, zz]
-                  end
-                }.compact
-              end)
+                  found.concat(q.empty? ? list :
+                               matcher.match(list, q, q[0, cx], q[cx..-1]))
                 end
                 next if skip
                 @sort ? found : found.reverse
               end
 
-        mcount = matches.length
-        if @sort && mcount <= @sort
-          matches.replace matches.sort_by { |pair|
-            line, offset = pair
-            [offset.last - offset.first, line.length, line]
-          }
+            if !empty && @sort && matches.length <= @sort
+              matches = sort_by_rank(matches)
             end
+
+            # Atomic update
+            @matches.set matches
           end#new_search
 
           # This small delay reduces the number of partial lists
-      if started && !user_input
-        sleep 0.2
-      end
-      started = true
+          sleep((delay = [20, delay + 5].min) * 0.01) unless user_input
 
-      if events.delete(:vcursor) || new_search
-        @mtx.synchronize do
-          plcount  = [@matches.length, max_items].min
-          @matches = matches
-          vcursor  = @vcursor = [0, [@vcursor, mcount - 1, max_items - 1].min].max
+          update_list new_search
+        end#while
+      rescue Exception => e
+        @main.raise e
       end
+    }
   end
 
-      # Output
-      @smtx.synchronize do
-        item_length = [mcount, max_items].min
-        if item_length < plcount
-          plcount.downto(item_length) do |idx|
+  def pick
+    items = @matches[0, max_items]
+    curr = [0, [@vcursor.get, items.length - 1].min].max
+    [*items.fetch(curr, [])][0]
+  end
+
+  def update_list wipe
+    render do
+      items = @matches[0, max_items]
+
+      # Wipe
+      if items.length < @plcount
+        @plcount.downto(items.length) do |idx|
           C.setpos cursor_y - idx - 2, 0
           C.clrtoeol
         end
       end
+      @plcount = items.length
 
-        maxc = C.cols - 5
-        matches[0, max_items].each_with_index do |item, idx|
-          next if !new_search && !((vcursor-1)..(vcursor+1)).include?(idx)
-
-          line, offset = convert_item item
+      maxc    = C.cols - 3
+      vcursor = @vcursor.set { |v| [0, [v, items.length - 1].min].max }
+      cleanse = Set[vcursor]
+      @vcursors.set { |vs|
+        cleanse.merge vs
+        Set.new
+      }
+      items.each_with_index do |item, idx|
+        next unless wipe || cleanse.include?(idx)
         row           = cursor_y - idx - 2
         chosen        = idx == vcursor
-          b, e = offset
-
-          if line.length > maxc
-            diff = e - (maxc - 2)
-            if diff > 2
-              line = '..' + line[diff..-1]
-              b -= diff - 2
-              b = [2, b].max
-            else
-              line = line[0, maxc] + '..'
+        selected      = @selects.include?([*item][0])
+        line, offsets = convert_item item
+        tokens        = format line, maxc, offsets
+        print_item row, tokens, chosen, selected
+      end
+      print_info
     end
   end
-          if line.length > maxc
-            line = line[0, maxc] + '..'
-          end
 
-          C.setpos row, 0
-          C.clrtoeol
-          cprint chosen ? '>' : ' ', color(:red, true)
-          cprint ' ', chosen ? color(:chosen) : 0
-
-          C.attron color(:chosen, true) if chosen
-
-          e = [e, maxc].min
-          if b < maxc && b < e
-            C.addstr line[0, b]
-            cprint   line[b...e], color(chosen ? :match! : :match, chosen)
-            C.attron color(:chosen, true) if chosen
-            C.addstr line[e..-1] || ''
-          else
-            C.addstr line
-          end
-          C.attroff color(:chosen, true) if chosen
-        end
-
-        print_info if !@lists.empty? || events[:loaded]
-        print_input
+  def start_renderer
+    Thread.new do
+      begin
+        while blk = @queue.shift
+          blk.call
           refresh
         end
-    end
       rescue Exception => e
-    main.raise e
+        @main.raise e
+      end
+    end
   end
-}
 
+  def render &blk
+    @queue.push blk
+    nil
+  end
+
+  def start_loop
     got = nil
     begin
       tty     = IO.open(IO.sysopen('/dev/tty'), 'r')
       input   = ''
       cursor  = 0
+      backword = proc {
+        cursor = (input[0, cursor].rindex(/\s\S/) || -1) + 1
+      }
       actions = {
-    :nop     => proc {},
+        :nop     => proc { nil },
         ctrl(:c) => proc { exit 1 },
         ctrl(:d) => proc { exit 1 if input.empty? },
         ctrl(:m) => proc {
-      @mtx.synchronize do
-        got = @matches.fetch(@vcursor, [])[0]
-      end
+          got = pick
           exit 0
         },
         ctrl(:u) => proc { input = input[cursor..-1]; cursor = 0 },
-    ctrl(:a) => proc { cursor = 0 },
-    ctrl(:e) => proc { cursor = input.length },
-    ctrl(:j) => proc { emit(:vcursor) { @vcursor -= 1 } },
-    ctrl(:k) => proc { emit(:vcursor) { @vcursor += 1 } },
+        ctrl(:a) => proc { cursor = 0; nil },
+        ctrl(:e) => proc { cursor = input.length; nil },
+        ctrl(:j) => proc { @vcursor.set { |v| @vcursors << v; v - 1 }; update_list false },
+        ctrl(:k) => proc { @vcursor.set { |v| @vcursors << v; v + 1 }; update_list false },
         ctrl(:w) => proc {
-      ridx   = (input[0...cursor - 1].rindex(/\S\s/) || -2) + 2
-      input  = input[0...ridx] + input[cursor..-1]
-      cursor = ridx
+          pcursor = cursor
+          backword.call
+          input = input[0...cursor] + input[pcursor..-1]
         },
         127 => proc { input[cursor -= 1] = '' if cursor > 0 },
-    :left => proc { cursor = [0, cursor - 1].max },
-    :right => proc { cursor = [input.length, cursor + 1].min },
+        9   => proc { |o|
+          if @multi && sel = pick
+            if @selects.has_key? sel
+              @selects.delete sel
+            else
+              @selects[sel] = 1
+            end
+            @vcursor.set { |v|
+              @vcursors << v
+              v + (o == :stab ? 1 : -1)
+            }
+            update_list false
+          end
+        },
+        :left  => proc { cursor = [0, cursor - 1].max; nil },
+        :right => proc { cursor = [input.length, cursor + 1].min; nil },
+        :alt_b => proc { backword.call; nil },
+        :alt_f => proc {
+          cursor += (input[cursor..-1].index(/(\S\s)|(.$)/) || -1) + 1
+          nil
+        },
       }
       actions[ctrl(:b)] = actions[:left]
       actions[ctrl(:f)] = actions[:right]
       actions[ctrl(:h)] = actions[127]
       actions[ctrl(:n)] = actions[ctrl(:j)]
       actions[ctrl(:p)] = actions[ctrl(:k)]
+      actions[:stab]    = actions[9]
 
       while true
+        @cursor_x.set cursor
+        render { print_input }
+
         ord = tty.getc.ord
-    if ord == 27
-      ord = tty.getc.ord
-      if ord == 91
-        ord = case tty.getc.ord
+        ord =
+          case ord = tty.getc.ord
+          when 91
+            case tty.getc.ord
             when 68 then :left
             when 67 then :right
             when 66 then ctrl(:j)
             when 65 then ctrl(:k)
+            when 90 then :stab
             else :nop
             end
-      end
-    end
+          when 'b'.ord
+            :alt_b
+          when 'f'.ord
+            :alt_f
+          else
+            ord
+          end if ord == 27
 
-    actions.fetch(ord, proc { |ord|
+        upd = actions.fetch(ord, proc { |ord|
           char = [ord].pack('U*')
           if char =~ /[[:print:]]/
             input.insert cursor, char
@@ -530,17 +715,135 @@ begin
         }).call(ord)
 
         # Dispatch key event
-    emit(:key) { @query = input.dup }
-
-    # Update user input
-    @smtx.synchronize do
-      @cursor_x = cursor
-      print_input
-      refresh
-    end
+        emit(:key) { [@query.set(input.dup), cursor] } if upd
       end
     ensure
       C.close_screen
-  stdout.puts got if got
+      if got
+        if @selects.empty?
+          @stdout.puts got
+        else
+          @selects.each do |sel, _|
+            @stdout.puts sel
+          end
+        end
+      end
+    end
   end
 
+  class FuzzyMatcher < Matcher
+    attr_reader :caches, :rxflag
+
+    def initialize rxflag
+      @caches = Hash.new { |h, k| h[k] = {} }
+      @regexp = {}
+      @rxflag = rxflag
+    end
+
+    def empty? q
+      q.empty?
+    end
+
+    def fuzzy_regex q
+      @regexp[q] ||= begin
+        q = q.downcase if @rxflag != 0
+        Regexp.new(convert_query(q).inject('') { |sum, e|
+          e = Regexp.escape e
+          sum << "#{e}[^#{e}]*?"
+        }, @rxflag)
+      end
+    end
+
+    def match list, q, prefix, suffix
+      regexp = fuzzy_regex q
+
+      cache = @caches[list.object_id]
+      prefix_cache = nil
+      (prefix.length - 1).downto(1) do |len|
+        break if prefix_cache = cache[prefix[0, len]]
+      end
+
+      suffix_cache = nil
+      0.upto(suffix.length - 1) do |idx|
+        break if suffix_cache = cache[suffix[idx..-1]]
+      end unless suffix.empty?
+
+      partial_cache = [prefix_cache,
+                       suffix_cache].compact.sort_by { |e| e.length }.first
+      cache[q] ||= (partial_cache ?
+                    partial_cache.map { |e| e.first } : list).map { |line|
+        # Ignore errors: e.g. invalid byte sequence in UTF-8
+        md = line.match(regexp) rescue nil
+        md && [line, [md.offset(0)]]
+      }.compact
+    end
+  end
+
+  class ExtendedFuzzyMatcher < FuzzyMatcher
+    def initialize rxflag
+      super
+      @regexps = {}
+    end
+
+    def empty? q
+      parse(q).empty?
+    end
+
+    def parse q
+      q = q.strip
+      @regexps[q] ||= q.split(/\s+/).map { |w|
+        invert =
+          if w =~ /^!/
+            w = w[1..-1]
+            true
+          end
+
+        [ @regexp[w] ||=
+            case w
+            when ''
+              nil
+            when /^'/
+              w.length > 1 ?
+                Regexp.new(Regexp.escape(w[1..-1]), rxflag) : nil
+            when /^\^/
+              w.length > 1 ?
+                Regexp.new('^' << Regexp.escape(w[1..-1]), rxflag) : nil
+            when /\$$/
+              w.length > 1 ?
+                Regexp.new(Regexp.escape(w[0..-2]) << '$', rxflag) : nil
+            else
+              fuzzy_regex w
+            end, invert ]
+      }.select { |pair| pair.first }
+    end
+
+    def match list, q, prefix, suffix
+      regexps = parse q
+      # Look for prefix cache
+      cache  = @caches[list.object_id]
+      prefix = prefix.strip.sub(/\$\S*$/, '').sub(/(^|\s)!\S*$/, '')
+      prefix_cache = nil
+      (prefix.length - 1).downto(1) do |len|
+        break if prefix_cache = cache[Set[@regexps[prefix[0, len]]]]
+      end
+
+      cache[Set[regexps]] ||= (prefix_cache ?
+                               prefix_cache.map { |e| e.first } :
+                               list).map { |line|
+        offsets = []
+        regexps.all? { |pair|
+          regexp, invert = pair
+          md = line.match(regexp) rescue nil
+          if md && !invert
+            offsets << md.offset(0)
+          elsif !md && invert
+            true
+          end
+        } && [line, offsets]
+      }.select { |e| e }
+    end
+  end
+end#FZF
+
+FZF.new(ARGV, $stdin).start if ENV.fetch('FZF_EXECUTABLE', '1') == '1'
+

fzf.gemspec

@@ -1,7 +1,7 @@
 # coding: utf-8
 Gem::Specification.new do |spec|
   spec.name          = 'fzf'
-  spec.version       = '0.1.0'
+  spec.version       = '0.4.0'
   spec.authors       = ['Junegunn Choi']
   spec.email         = ['junegunn.c@gmail.com']
   spec.description   = %q{Fuzzy finder for your shell}

plugin/fzf.vim

@@ -27,9 +27,13 @@ function! s:fzf(args)
   try
     let tf = tempname()
     let prefix = exists('g:fzf_command') ? g:fzf_command.'|' : ''
-    execute "silent !".prefix."/usr/bin/env ruby ".s:exec." ".a:args." > ".tf
+    let fzf = executable(s:exec) ? s:exec : 'fzf'
+    execute "silent !".prefix.fzf." ".a:args." > ".tf
     if !v:shell_error
-      execute 'silent e '.join(readfile(tf), '')
+      let file = join(readfile(tf), '')
+      if !empty(file)
+        execute 'silent e '.file
+      endif
     endif
   finally
     silent! call delete(tf)

test/test_fzf.rb

@@ -0,0 +1,306 @@
+#!/usr/bin/env ruby
+# encoding: utf-8
+
+require 'minitest/autorun'
+$LOAD_PATH.unshift File.expand_path('../..', __FILE__)
+ENV['FZF_EXECUTABLE'] = '0'
+load 'fzf'
+
+class TestFZF < MiniTest::Unit::TestCase
+  def test_default_options
+    fzf = FZF.new []
+    assert_equal 1000, fzf.sort
+    assert_equal false, fzf.multi
+    assert_equal true, fzf.color
+    assert_equal Regexp::IGNORECASE, fzf.rxflag
+
+    begin
+      ENV['FZF_DEFAULT_SORT'] = '1500'
+      fzf = FZF.new []
+      assert_equal 1500, fzf.sort
+    ensure
+      ENV.delete 'FZF_DEFAULT_SORT'
+    end
+  end
+
+  def test_option_parser
+    # Long opts
+    fzf = FZF.new %w[--sort=2000 --no-color --multi +i]
+    assert_equal 2000, fzf.sort
+    assert_equal true, fzf.multi
+    assert_equal false, fzf.color
+    assert_equal 0, fzf.rxflag
+
+    # Short opts
+    fzf = FZF.new %w[-s 2000 +c -m +i]
+    assert_equal 2000, fzf.sort
+    assert_equal true, fzf.multi
+    assert_equal false, fzf.color
+    assert_equal 0, fzf.rxflag
+  end
+
+  def test_invalid_option
+    [%w[-s 2000 +s], %w[yo dawg]].each do |argv|
+      assert_raises(SystemExit) do
+        fzf = FZF.new argv
+      end
+    end
+  end
+
+  # FIXME Only on 1.9 or above
+  def test_width
+    fzf = FZF.new []
+    assert_equal 5, fzf.width('abcde')
+    assert_equal 4, fzf.width('한글')
+    assert_equal 5, fzf.width('한글.')
+  end
+
+  def test_trim
+    fzf = FZF.new []
+    assert_equal ['사.',     6], fzf.trim('가나다라마바사.', 4, true)
+    assert_equal ['바사.',   5], fzf.trim('가나다라마바사.', 5, true)
+    assert_equal ['바사.',   5], fzf.trim('가나다라마바사.', 6, true)
+    assert_equal ['마바사.', 4], fzf.trim('가나다라마바사.', 7, true)
+    assert_equal ['가나',    6], fzf.trim('가나다라마바사.', 4, false)
+    assert_equal ['가나',    6], fzf.trim('가나다라마바사.', 5, false)
+    assert_equal ['가나a',   6], fzf.trim('가나ab라마바사.', 5, false)
+    assert_equal ['가나ab',  5], fzf.trim('가나ab라마바사.', 6, false)
+    assert_equal ['가나ab',  5], fzf.trim('가나ab라마바사.', 7, false)
+  end
+
+  def test_format
+    fzf = FZF.new []
+    assert_equal [['01234..', false]], fzf.format('0123456789', 7, [])
+    assert_equal [['012', false], ['34', true], ['..', false]],
+      fzf.format('0123456789', 7, [[3, 5]])
+    assert_equal [['..56', false], ['789', true]],
+      fzf.format('0123456789', 7, [[7, 10]])
+    assert_equal [['..56', false], ['78', true], ['9', false]],
+      fzf.format('0123456789', 7, [[7, 9]])
+
+    (3..5).each do |i|
+      assert_equal [['..', false], ['567', true], ['89', false]],
+        fzf.format('0123456789', 7, [[i, 8]])
+    end
+
+    assert_equal [['..', false], ['345', true], ['..', false]],
+      fzf.format('0123456789', 7, [[3, 6]])
+    assert_equal [['012', false], ['34', true], ['..', false]],
+      fzf.format('0123456789', 7, [[3, 5]])
+
+    # Multi-region
+    assert_equal [["0", true], ["1", false], ["2", true], ["34..", false]],
+      fzf.format('0123456789', 7, [[0, 1], [2, 3]])
+
+    assert_equal [["..", false], ["5", true], ["6", false], ["78", true], ["9", false]],
+      fzf.format('0123456789', 7, [[3, 6], [7, 9]])
+
+    assert_equal [["..", false], ["3", true], ["4", false], ["5", true], ["..", false]],
+      fzf.format('0123456789', 7, [[3, 4], [5, 6]])
+
+    # Multi-region Overlap
+    assert_equal [["..", false], ["345", true], ["..", false]],
+      fzf.format('0123456789', 7, [[4, 5], [3, 6]])
+  end
+
+  def test_fuzzy_matcher
+    matcher = FZF::FuzzyMatcher.new Regexp::IGNORECASE
+    list = %w[
+      juice
+      juiceful
+      juiceless
+      juicily
+      juiciness
+      juicy]
+    assert matcher.caches.empty?
+    assert_equal(
+      [["juice",     [[0, 1]]],
+       ["juiceful",  [[0, 1]]],
+       ["juiceless", [[0, 1]]],
+       ["juicily",   [[0, 1]]],
+       ["juiciness", [[0, 1]]],
+       ["juicy",     [[0, 1]]]], matcher.match(list, 'j', '', '').sort)
+    assert !matcher.caches.empty?
+    assert_equal [list.object_id], matcher.caches.keys
+    assert_equal 1, matcher.caches[list.object_id].length
+    assert_equal 6, matcher.caches[list.object_id]['j'].length
+
+    assert_equal(
+      [["juicily",   [[0, 5]]],
+       ["juiciness", [[0, 5]]]], matcher.match(list, 'jii', '', '').sort)
+
+    assert_equal(
+      [["juicily",   [[2, 5]]],
+       ["juiciness", [[2, 5]]]], matcher.match(list, 'ii', '', '').sort)
+
+    assert_equal 3, matcher.caches[list.object_id].length
+    assert_equal 2, matcher.caches[list.object_id]['ii'].length
+
+    # TODO : partial_cache
+  end
+
+  def test_fuzzy_matcher_case_sensitive
+    assert_equal [['Fruit', [[0, 5]]]],
+      FZF::FuzzyMatcher.new(0).match(%w[Fruit Grapefruit], 'Fruit', '', '').sort
+
+    assert_equal [["Fruit", [[0, 5]]], ["Grapefruit", [[5, 10]]]],
+      FZF::FuzzyMatcher.new(Regexp::IGNORECASE).
+      match(%w[Fruit Grapefruit], 'Fruit', '', '').sort
+  end
+
+  def test_extended_fuzzy_matcher
+    matcher = FZF::ExtendedFuzzyMatcher.new Regexp::IGNORECASE
+    list = %w[
+      juice
+      juiceful
+      juiceless
+      juicily
+      juiciness
+      juicy
+      _juice]
+    match = proc { |q, prefix|
+      matcher.match(list, q, prefix, '').sort.map { |p| [p.first, p.last.sort] }
+    }
+
+    assert matcher.caches.empty?
+    3.times do
+      ['y j', 'j y'].each do |pat|
+        (0..pat.length - 1).each do |prefix_length|
+          prefix = pat[0, prefix_length]
+          assert_equal(
+            [["juicily", [[0, 1], [6, 7]]],
+             ["juicy",   [[0, 1], [4, 5]]]],
+            match.call(pat, prefix))
+        end
+      end
+
+      # $
+      assert_equal [["juiceful",  [[7, 8]]]], match.call('l$', '')
+      assert_equal [["juiceful",  [[7, 8]]],
+                    ["juiceless", [[5, 6]]],
+                    ["juicily",   [[5, 6]]]], match.call('l', '')
+
+      # ^
+      assert_equal list.length,     match.call('j', '').length
+      assert_equal list.length - 1, match.call('^j', '').length
+
+      # !
+      assert_equal 0, match.call('!j', '').length
+
+      # ! + ^
+      assert_equal [["_juice", []]], match.call('!^j', '')
+
+      # ! + $
+      assert_equal list.length - 1, match.call('!l$', '').length
+
+      # ! + f
+      assert_equal [["juicy", [[4, 5]]]], match.call('y !l', '')
+
+      # '
+      assert_equal %w[juiceful juiceless juicily],
+        match.call('il', '').map { |e| e.first }
+      assert_equal %w[juicily],
+        match.call("'il", '').map { |e| e.first }
+      assert_equal (list - %w[juicily]).sort,
+        match.call("!'il", '').map { |e| e.first }.sort
+    end
+    assert !matcher.caches.empty?
+  end
+
+  def test_xfuzzy_matcher_prefix_cache
+    matcher = FZF::ExtendedFuzzyMatcher.new Regexp::IGNORECASE
+    list = %w[
+      a.java
+      b.java
+      java.jive
+      c.java$
+      d.java
+    ]
+    2.times do
+      assert_equal 5, matcher.match(list, 'java',   'java',   '').length
+      assert_equal 3, matcher.match(list, 'java$',  'java$',  '').length
+      assert_equal 1, matcher.match(list, 'java$$', 'java$$', '').length
+
+      assert_equal 0, matcher.match(list, '!java',  '!java',  '').length
+      assert_equal 4, matcher.match(list, '!^jav',  '!^jav',  '').length
+      assert_equal 4, matcher.match(list, '!^java', '!^java', '').length
+      assert_equal 2, matcher.match(list, '!^java !b !c', '!^java', '').length
+    end
+  end
+
+  def test_sort_by_rank
+    matcher  = FZF::FuzzyMatcher.new Regexp::IGNORECASE
+    xmatcher = FZF::ExtendedFuzzyMatcher.new Regexp::IGNORECASE
+    list = %w[
+      0____1
+      0_____1
+      01
+      ____0_1
+      01_
+      _01_
+      0______1
+      ___01___
+    ]
+    assert_equal(
+      [["01",       [[0, 2]]],
+       ["01_",      [[0, 2]]],
+       ["_01_",     [[1, 3]]],
+       ["___01___", [[3, 5]]],
+       ["____0_1",  [[4, 7]]],
+       ["0____1",   [[0, 6]]],
+       ["0_____1",  [[0, 7]]],
+       ["0______1", [[0, 8]]]],
+      FZF.new([]).sort_by_rank(matcher.match(list, '01', '', '')))
+
+    assert_equal(
+      [["01",       [[0, 1], [1, 2]]],
+       ["01_",      [[0, 1], [1, 2]]],
+       ["_01_",     [[1, 2], [2, 3]]],
+       ["0____1",   [[0, 1], [5, 6]]],
+       ["0_____1",  [[0, 1], [6, 7]]],
+       ["____0_1",  [[4, 5], [6, 7]]],
+       ["0______1", [[0, 1], [7, 8]]],
+       ["___01___", [[3, 4], [4, 5]]]],
+      FZF.new([]).sort_by_rank(xmatcher.match(list, '0 1', '', '')))
+
+    assert_equal(
+      [["_01_",     [[1, 3], [0, 4]]],
+       ["0____1",   [[0, 6], [1, 3]]],
+       ["0_____1",  [[0, 7], [1, 3]]],
+       ["0______1", [[0, 8], [1, 3]]],
+       ["___01___", [[3, 5], [0, 2]]],
+       ["____0_1",  [[4, 7], [0, 2]]]],
+      FZF.new([]).sort_by_rank(xmatcher.match(list, '01 __', '', '')))
+  end
+
+  if RUBY_PLATFORM =~ /darwin/
+    NFD = '한글'
+    def test_nfc
+      assert_equal 6, NFD.length
+      assert_equal ["한글", [[0, 1], [1, 2]]],
+        FZF::UConv.nfc(NFD, [[0, 3], [3, 6]])
+
+      nfd2 = 'before' + NFD + 'after'
+      assert_equal 6 + 6 + 5, nfd2.length
+
+      nfc, offsets = FZF::UConv.nfc(nfd2, [[4, 14], [9, 13]])
+      o1, o2 = offsets
+      assert_equal 'before한글after', nfc
+      assert_equal 're한글af',        nfc[(o1.first...o1.last)]
+      assert_equal '글a',             nfc[(o2.first...o2.last)]
+    end
+
+    def test_nfd
+      nfc = '한글'
+      nfd = FZF::UConv.nfd(nfc)
+      assert_equal 6, nfd.length
+      assert_equal NFD, nfd
+    end
+  end
+
+  def test_split
+    assert_equal ["a", "b", "c", "\xFF", "d", "e", "f"],
+      FZF::UConv.split("abc\xFFdef")
+  end
+end
+