fix: improve matchfuzzy() scoring for prefix matches versus CamelCase - #16797

Thanks for working on it!

I am not sure if an option is the best way to handle it to be honest.

If current behavior considered counter-intuitive (which I agree with) then wouldn't the best way to handle it -- make it intuitive?

Not really. Because camel case is used for extra points. This works in most languages. And there is no option to control the extra points behavior. So can only do this.. In short, it is useful when your style requires camelCase. It is not so useful when you use fuzzy in some tools.

But even in camelCase, I would still expect to rank things higher than thisThings giving thin as input.

as in here:

echo matchfuzzy(['thisThings', 'things'], 'thin')
# result is ['thisThings', 'things']
# I would expect ['things', 'thisThings']

sThin higher than thin .

sThin higher than thin .

compare following results:

echo matchfuzzy(['thisThings', 'things', 'sThings'], 'things')
# ['things', 'sThings', 'thisThings']

echo matchfuzzy(['thisThings', 'things', 'sThings'], 'thing')
# ['sThings', 'thisThings', 'things']

For the first one I remember you did the fix without adding additional option.

In the second case I don't understand why sThings would be higher than things.

My understanding is that match at beginning of the string should be considered higher than match somewhere in the middle even if it is camelCase.

I mean if camelCase is more important than match at beginning then I would just accept it and let's not introduce a new option.

[['thisThings', 'things'], [[4, 5, 6, 7], [0, 1, 2, 3]], [334, 333]]

CamelCase here just checks if the letter before T is lowercase. If it is, it will be camel cased, so it will get one point. s is lowercase, so it gets one point

[['thisThings', 'things'], [[4, 5, 6, 7], [0, 1, 2, 3]], [334, 333]]

CamelCase here just checks if the letter before T is lowercase. If it is, it will be camel cased, so it will get one point. s is lowercase, so it gets one point

Yes I get it. What I don't get why we can't check if letter before T is none and add 1 point as well the same way camelCase adds it?

idk it's from the fts port. I think it's probably reasonable in most cases from the programming language naming perspective. It's a bit weird outside of other scenarios. So we need some fields to control the granularity of the score. For example camelcase..
Of course, we can check neighbor here and get extra points. But I am not sure this is the expected behavior. Since we have fuzzy, there are no issue reports about the current behavior (except yours..it create in last year). or maybe I just didn't notice it.

For me it looks like CAMEL_CASE bonus is just too high:

score += CAMEL_BONUS * 2;

And comment about fuzzy speaks about it as well:

 * Single words care about consecutive matches but not separators or camel
 * case.
 *   Score starts at 100
 *   Matched letter: +0 points
 *   Unmatched letter: -1 point
 *   Consecutive match bonus: +15 points
 *   First letter bonus: +15 points
 *   Separator bonus: +30 points
 *   Camel case bonus: +30 points
 *   Unmatched leading letter: -5 points (max: -15)

Single words care about consecutive matches but not separators or camel case

This might be lost in implementation?

if I do the same bonus for first match as for camelCase, then it works like I think it should:

// bonus if the first letter is matched
#define FIRST_LETTER_BONUS 60

consecutive_camel is something I added in my previous PR. Modifying FIRST_LETTER_BONUS is not the right approach.

I think we need to control the granularity of the bonus points. Secondly, we should also consider whether the words are matched continuously from the beginning. There should be extra bonus points. I will try to implement it later.

consecutive_camel is something I added in my previous PR. Modifying FIRST_LETTER_BONUS is not the right approach.

It is just an example of adding the same bonus as CAMEL_CASE to the match at start.

I think we need to control the granularity of the bonus points. Secondly, we should also consider whether the words are matched continuously from the beginning. There should be extra bonus points. I will try to implement it later.

Thanks!

It is just an example of adding the same bonus as CAMEL_CASE to the match at start.

It would probably be better to add same bonus for match at start of the word as for camelCase, wouldn't it?

idea is to give extra points to consecutive words, for example, the score of 3 consecutive positions is greater than 2 consecutive positions. may have to try it out to know the specific behavior.

idea is to give extra points to consecutive words, for example, the score of 3 consecutive positions is greater than 2 consecutive positions. may have to try it out to know the specific behavior.

maybe, although for me this looks tempting:

	    // Enhanced camel case scoring
	    if (vim_islower(neighbor) && vim_isupper(curr)) `or neighbor is absent` <--- here?
	    {
		score += CAMEL_BONUS * 2;  // Double the camel case bonus
		is_camel = TRUE;
		consecutive_camel++;
		// Additional bonus for consecutive camel
		if (consecutive_camel > 1)
		    score += CAMEL_BONUS;
	    }

if (vim_islower(neighbor) && vim_isupper(curr)) or neighbor is absent <--- here?

yep it works and is pretty simple. But I guess the problem is the lack of bonus points for consecutive matches like there are for exact matches.

@habamax I have updated the algorithm here. How do you feel about it ?

@glepnir looks promising

Thank you!

note that now following sorts sThings and thisThings differently

echo matchfuzzy(['thisThings', 'things', 'sThings'], 'things')
# ['things', 'sThings', 'thisThings'] -- WAS
# ['things', 'thisThings', 'sThings'] -- NOW

I would assume shorter string should have higher rank?

Sry... I made a mistake.

change to 60 this not works matchfuzzypos(['things', 'thisThings', 'sThings'], 'thin') .

I think it is necessary to control the scoring behavior when we use this search tool. Add fields to matchfuzzy. Secondly, the current algorithm. When 3 consecutive let consecutive affect camelcase..

change to 60 this not works matchfuzzypos(['things', 'thisThings', 'sThings'], 'thin') .

I think it is necessary to control the scoring behavior when we use this search tool. Add fields to matchfuzzy. Secondly, the current algorithm. When 3 consecutive let consecutive affect camelcase..

Why? It works fine...

Oh this is wrong matchfuzzy(['Cursor', 'lCursor', 'shCurlyIn', 'shCurlyError', 'TracesCursor', 'CurSearch', 'CursorLine'], 'Cur')

command line..script /Users/mw/workspace/vim/src/testdir/runtest.vim[620]..function RunTheTest[60]..Test_matchfuzzy line 53: Expected ['Cursor', 'CurSearch', 'CursorLine', 'lCursor', 'shCurlyIn', 'shCurlyError', 'TracesCursor'] but got ['lCursor', 'Cursor', 'shCurlyIn', 'CurSearch', 'shCurlyError', 'CursorLine', 'TracesCursor']

matchfuzzy(['Cursor', 'lCursor', 'shCurlyIn', 'shCurlyError', 'TracesCursor', 'CurSearch', 'CursorLine'], 'Cur')

Well, apparently, because sorting is changed, some of the test cases would be wrong. And the weight for first letter is the subject for discussion. Maybe if we make is a bit higher than camelCase, say 65, it would be sorted better?

Like in this case result would be:

echo matchfuzzy(['Cursor', 'lCursor', 'shCurlyIn', 'shCurlyError', 'TracesCursor', 'CurSearch', 'CursorLine'], 'Cur')

# lCursor, Cursor, CurSearch, CursorLine, shCurlyIn, shCurlyError, TracesCursor

But I would leave it with 60, same as what is used for camelCase

I feel the problem is not here... The bonus points need to be controlled in different usage scenarios. So why can't camelcase be used in matchfuzzy as a field? You use it in the tool without considering CamelCase, right? Secondly, I think the current algorithm is also OK. Because the tool input sequence score is changing. The input reading accuracy will also be higher, right? So when the order of 1 or 2 characters is wrong...

I think you might be right and we need to control it with an option/parameter to the function. The way you did it initially.

Yes. the default camelcase bouns are weird and cause a lot of strange visual effects.

Yes. the default camelcase bouns are weird and cause a lot of strange visual effects.

Agree, and we probably wouldn't be able to easily fix/improve it to not interfere with other things.

So yeah, I think now that your initial approach was correct.

I have just tested it with camelcase: false dict parameter and it works how I think it should work. Thank you!

thanks

I recently encountered a similar problem. I was also confused by the behavior of matchseq. After finishing other PRs