Use DidYouMean::SpellChecker for gem suggestions in Bundler

[austinpray]

• Similar to Improve spell checking suggestion performance by vendoringDidYouMean::Levenshtein.distance from did_you_mean-1.4.0 #3856
• I'll squash this when the changes settle down

---

Description:

replaces Bundler::SimilarityDetector with DidYouMean::SpellChecker

1. Higher quality spelling suggestions in my testing
2. Benefit from upstream improvements, less code to maintain in this repo. The current Bundler::SimilarityDetector.levenshtein_distance dynamic programming implementation could be optimized quite a bit.
3. Should see a significant performance improvement, under the hood DidYouMean::SpellChecker is highly optimized.

RubyGems 3.0 supports Ruby 2.3 or later. Ruby 2.3 and later ships with the did_you_mean gem.

https://stdgems.org/did_you_mean/

Higher quality spelling suggestions

The algorithm seems to

Tested with the rails 6 gemfile. Here's some results from manually stressing the spell checker:

input	old	new
regexparser	(none)	Did you mean 'regexp_parser'?
miniportile	(none)	Did you mean 'mini_portile2'?
rails-sanitizer-html	(none)	Did you mean 'rails-html-sanitizer'?
sqilte3	Did you mean sqlite3?	Did you mean 'sqlite3'?
sqte3	(none)	Did you mean 'sqlite3'?

Benchmark

💹 Benchmark code (permalink)

This looks like a 7x performance boost. The DidYouMean implementation also uses less memory and such.

Warming up --------------------------------------
              Before    12.000  i/100ms
               After    94.000  i/100ms
Calculating -------------------------------------
              Before    124.193  (± 1.6%) i/s -    624.000  in   5.025368s
               After    937.282  (± 0.9%) i/s -      4.700k in   5.014849s

Notes for reviewer

• Bundler::SimilarityDetector is not used in Bundler anymore. I would like to delete it, but I haven't haven't because I am not sure if we should pull the rug out from people that might be using for their own purposes. What should we do to phase out this module?

What was the end-user or developer problem that led to this PR?

Was improving performance over in #3856 and saw this as an opportunity as well.

What is your fix for the problem, implemented in this PR?

Use DidYouMean::SpellChecker as a drop-in replacement for Bundler::SimilarityDetector

---

Tasks:

• Describe the problem / feature
• Write tests
• Write code to solve the problem
• Get code review from coworkers / friends

I will abide by the code of conduct.

[austinpray]

Suggested change

	[words[0..-2].join(", "), words[-1]].join(" or ")
	[words[0..-2].join(", "), words[-1]].join(", or ")

Personally I like oxford commas ("apple, orange, or pear"). I went without it so the output is similar to how it has always been ("apple, orange or pear").

[austinpray]

I left this module in here so I don't pull the rug out from someone who might be using it in their own project. I would like to delete it, is there a deprecation path for stuff like this?

[austinpray]

Same concern as #3856 (review)

[hsbt]

@austinpray Can you update this Pr by rubocop -a bundler/lib/bundler/cli/common.rb?

[hsbt]

@austinpray Sorry to my late action. I'm happy to introduce this feature at Bundler 4 and Ruby 4.0. Thank you!