Added automatic character range detection that determines font - #3155

If you think this is ready for review, I can take a look at the changes.

Thanks ... I'm doing some last minute testing to see if I can squeeze out a bit more performance. Plus I need to update the render tests. But I think the basic logic is ready for review so have at it.

@robvanderlee FYI I rewrote the history to make this change a bit easier to follow.

There will always be a performance impact with this change since we're a) parsing each character of a text frame, and b) generating more frames (when required by font matching). I don't have a clean system for performance testing, but what testing I did do shows minimal impact on most documents. The greatest impact would be in the target scenario where the user just wants to specify a sequence of fonts and let Dompdf pick the right one. But even there the impact is not significant.

I tested with both PHP 7.1 and 8.2 using the current release of Dompdf and this branch with each of the backend options. The document I used was fairly simple in structure. Times below are in milliseconds.

The base document looks like this:

<!doctype html>
<html>
    <head>
        <link rel="preconnect" href="/api/browser/proxy?url=https%3A%2F%2Ffonts.googleapis.com">
        <link rel="preconnect" href="/api/browser/proxy?url=https%3A%2F%2Ffonts.gstatic.com" crossorigin>
        <link href="/api/browser/proxy?url=https%3A%2F%2Ffonts.googleapis.com%2Fcss2%3Ffamily%3DSawarabi%2BGothic%26amp%3Bdisplay%3Dswap" rel="stylesheet">
        <style>
            @font-face {
                font-family: 'Great Vibes';
                font-style: normal;
                font-weight: 400;
                src: url('https://www.1001fonts.com/download/font/great-vibes.regular.ttf') format('truetype');
            }
        </style>
        <style>
            @font-face {
                font-family: 'Extraoin Zhurdlyou Anarchy';
                src: url('https://www.1001fonts.com/download/font/extraoin-zhurdlyou.anarchy.ttf') format('truetype');
                font-style: normal;
                font-weight: 400;
            }
        </style>
        <style>
            body {
                font-family: 'Great Vibes', 'Extraoin Zhurdlyou Anarchy', 'Sawarabi Gothic', serif;
            }
            .japanese {
                font-family: 'Sawarabi Gothic', serif;
            }
            .cyrillic {
                font-family: 'Extraoin Zhurdlyou Anarchy', serif;
                font-size: 32px;
                font-weight: 400;
            }
        </style>
    </head>
    <body>
        <!-- text goes here -->
    </body>
</html>

First test, 100 paragraphs of lorem ipsum:

Next test, a document with lots of manual font styling to render the text with the appropriate font. The document was 100 of the following paragraphs. This is not ideal since there is no text variability, but there aren't many lorem ipsum generators that work with multiple languages in the body content. Let alone segment the text appropriately.

<p><span class="japanese">能登</span> - <span class="cyrillic">Каждый</span> - <span class="japanese">半島 能登</span> - <span class="cyrillic">Каждый</span> - <span class="japanese">半島 能登</span> - This is great vibes font rendered text <span class="cyrillic">Каждый</span> - <span class="japanese">半島 能登</span> - <span class="cyrillic">Каждый</span> - <span class="japanese">半島 能登</span> - <span class="cyrillic">Каждый</span> - <span class="japanese">半島 能登</span></p>

Final test, the same as above without the any manual text styling.

<p>能登 - Каждый - 半島 能登 - Каждый - 半島 能登 - This is great vibes font rendered text Каждый - 半島 能登 - Каждый - 半島 能登 - Каждый - 半島</p>

As a quick note, the render tests fail because bold and italic font styles are not applied anymore.

Thank for finding that. An embarrassing oversight on my part for sure. I neglected to include the font style/weight as part of the font mapping logic. Should be OK now. I obviously was too focused on a particular test case. I'm sure I would have figured that out ... eventually.

Unit tests seem happy and most of the "core" test I have up in the debug helper look OK on first pass. I am seeing a slight difference in the rendering of some document (e.g., https://eclecticgeek.com/dompdf/core_tests/encoding_utf-8_w3.html) that I need to investigate.

Will look at the rest of the code later this week.

Unit tests seem happy and most of the "core" test I have up in the debug helper look OK on first pass. I am seeing a slight difference in the rendering of some document (e.g., https://eclecticgeek.com/dompdf/core_tests/encoding_utf-8_w3.html) that I need to investigate.

Found two issues.

• Non-printable characters weren't excluded from the font mapping. This was causing unnecessary frame splitting which was particularly problematic for preformatted (white-space: pre;) text.
• Frame splitting performed around font mapping exposed an issue where unsupported characters in a font were not being included in text width calculations when using Cpdf as the backend. This was causing frame overlap when a frame consisted entirely of unsupported characters for the used font.

The font-resolve logic is now essentially duplicated: We have the existing logic in the Style class and the new mapFontsToText, which also takes the current text into account. Some ideas on how to resolve this:

• Change the font-family logic in the Style class to compute the declaration to an array of the font names instead of the first available font. Maybe the final font paths, so non-existing fonts would be dropped already.
• The Text reflower would then set the final font name as the used value of the property, as it does now. Whether the property has been resolved could then be determined based on whether it returns an array or a string.
• The question would then be how to handle the other places where font-family is checked on non-text frames, e.g. when determining the height of inline frames. Might be right to use the first available font there, as it is essentially done now, but I haven’t verified that.

But maybe this does not need to be addressed right now, and can be tackled later.

One more thing: Shrink-to-fit width determination (for table columns, float, etc.) is now incorrect when text is split via the new logic, since get_min_max_width is called before reflow. That should be solvable by applying the split early if needed, but would need a way to determine whether the final font has been determined for a specific frame to avoid checking the text twice.

@bsweeney, @Mellthas great to see the improvements! I'll be sure to run some testing on my part and see If i can contribute to the review and/or make some suggested changes. I've had a busy couple weeks behind me but I'll make some time in this week to give a proper look.

One more thing: Shrink-to-fit width determination (for table columns, float, etc.) is now incorrect when text is split via the new logic, since get_min_max_width is called before reflow. That should be solvable by applying the split early if needed, but would need a way to determine whether the final font has been determined for a specific frame to avoid checking the text twice.

There appear to be two issues related to this?

1. Incorrect width determination since the used font may have differing widths than the first valid.
2. When a frame is split the new frame is pushed to the next line.

I think you're right that performing the font mapping earlier in the process is probably necessary. I'll need to look at this some more.

Update to address the shrink-to-fit issue by moving the font mapping logic to the decorator so that it can be called as necessary. Duplicated calls are avoided by recording the mapped font in a private property. I do think it would work better to handle recognition of mapping being complete by inspecting the used value ... somehow. I didn't see an obvious good way to do this so I'll leave that as an optimization for another time.