Repository navigation
lexer change: Allow emphasis in CJK text without spaces - #2648
Conversation
laurmaedje
left a comment
laurmaedje
left a comment
There was a problem hiding this comment.
Do we really need two large bold fonts in the repo for testing or would one suffice? I assume this is to test different scripts?
Actually in the test I added in this PR, only the sc-bold variant font is needed. But the irrelevant test
The outlines should be bold, but previously it is not I nobody noticed that because there is no bold CJK font at all. So, in summary:
So what do you prefer? Are you willing to accept the trade-off of having a test with broken font in order to save disk sapce? |
|
I guess we can keep both fonts. |
|
Thank you, I think this will really help CJK users! |
I would like to talk something I know about it. Traditionally, there is no concept of "bold" in Chinese publishing technology. When emphasizing, we usually replace a font type, such as regular script or imitation Song script. Therefore, Chinese font files often do not contain multiple font weights. However, the emergence of Microsoft Word allows people to use 'Fake Bold' to bold any character for emphasis, and many people have become accustomed to using bolder characters for emphasis. In this trend, some new fonts (such as Source Han) have also begun to release multiple font weights. It should be noted that emphasizing in bold is still not a common practice in official publications and is usually only done in (informal) electronic documents. In the Chinese macro set of LaTeX, the default behavior of the |
|
We already have a distinction between normal emphasis ( This means we could change the default behaviour of emph/strong based on the current text language. And users can of course also do it themselves with a show rule. |
|
There's may something wrong in |
|
This PR only ensured that emphasis is parsable in CJK text. Special handling for Kai font instead of italics is not yet implemented. |
Typst documentation says that `*` strong emphasis "only works at word boundaries." However, in typst/typst#2648 this was changed for CJK. Closes #49.
## 0.6.1 * Fix precedence for functions (#55). `1(x)` and `!(x)` should not be parsed as functions. Note that we still don't match typst's behavior for `f_"!"(x)`. For us this works just like `f_!(x)`, but for typst we get a function in the subscript for the former but not the latter. Fixing this would require some changes in the types. * Define sys.version and sys.inputs.typst-hs-version (#56). The former is set to the version of typst we are trying to implement. The latter is a stringified version number from typst-hs. This will allow typst programs to tell when they're running on typst-hs (or pandoc), and react accordingly. * Rename stBeforeSpace -> stSpaceBefore to avoid confusion. * Fix precedence issues in math parsing (#54). Increased precedence of ! (factorial). `_` or `^` should eagerly gobble a grouped argument (`c_(a)`). * Minimal support for `context` (#53). Parse `context` keyword. New Context constructor in Expr [API change]. Evaluate this by just evaluating the expression, for now. Note that we don't support the features (like location or numbering) that context is used to affect anyway, so this change probably won't be enough for meaningful support. But it might prevent some documents from issuing errors. * Arguments at method. * Array windows, reduce, to-dict methods. * Allow parentheses in import. * Make standard module available under std (typst 0.12). * Add over/underparen, over/undershell in math module. * Add stretch function. * Add skew. * Depend on typst-symbols 0.1.7 and start to target typst 0.12. * Reset indentation requirements inside `[]` content block. e.g. ``` / B: #block[ - a ] ``` We don't need indentation inside the block content. ## 0.6 * Recognize figure.caption function (#52). * Allow defined identifiers to override math defaults (#51). Previously we evaluated all math in a special environment that shadowed any user-defined functions with the same-named functions from the math or sym modules. This change gives user-defined identifiers priority over the math defaults, allowing things like `bb` to be overridden. * Typst.Types: EvalState now has two new fields, `evalMathIdentifiers` and `evalStandardIdentifiers`. `evalIdentifiers` is now just for user-defined identifiers. [API change] * Don't implicitly load sys module for math. ## 0.5.0.5 * Allow numbers like `1.` in math mode (#50). ## 0.5.0.4 * Add built-in identifiers for standard types (#21): array, bool, content, int, float, regex, length, alignment, color, symbol, string. * Adjust emphasis parser for CJK characters (#49). Typst documentation says that `*` strong emphasis "only works at word boundaries." However, in typst/typst#2648 this was changed for CJK.
|
Fullwidth alphanumerics are not covered by this approach. |
## 0.6.1 * Fix precedence for functions (NetBSD#55). `1(x)` and `!(x)` should not be parsed as functions. Note that we still don't match typst's behavior for `f_"!"(x)`. For us this works just like `f_!(x)`, but for typst we get a function in the subscript for the former but not the latter. Fixing this would require some changes in the types. * Define sys.version and sys.inputs.typst-hs-version (NetBSD#56). The former is set to the version of typst we are trying to implement. The latter is a stringified version number from typst-hs. This will allow typst programs to tell when they're running on typst-hs (or pandoc), and react accordingly. * Rename stBeforeSpace -> stSpaceBefore to avoid confusion. * Fix precedence issues in math parsing (NetBSD#54). Increased precedence of ! (factorial). `_` or `^` should eagerly gobble a grouped argument (`c_(a)`). * Minimal support for `context` (NetBSD#53). Parse `context` keyword. New Context constructor in Expr [API change]. Evaluate this by just evaluating the expression, for now. Note that we don't support the features (like location or numbering) that context is used to affect anyway, so this change probably won't be enough for meaningful support. But it might prevent some documents from issuing errors. * Arguments at method. * Array windows, reduce, to-dict methods. * Allow parentheses in import. * Make standard module available under std (typst 0.12). * Add over/underparen, over/undershell in math module. * Add stretch function. * Add skew. * Depend on typst-symbols 0.1.7 and start to target typst 0.12. * Reset indentation requirements inside `[]` content block. e.g. ``` / B: #block[ - a ] ``` We don't need indentation inside the block content. ## 0.6 * Recognize figure.caption function (NetBSD#52). * Allow defined identifiers to override math defaults (NetBSD#51). Previously we evaluated all math in a special environment that shadowed any user-defined functions with the same-named functions from the math or sym modules. This change gives user-defined identifiers priority over the math defaults, allowing things like `bb` to be overridden. * Typst.Types: EvalState now has two new fields, `evalMathIdentifiers` and `evalStandardIdentifiers`. `evalIdentifiers` is now just for user-defined identifiers. [API change] * Don't implicitly load sys module for math. ## 0.5.0.5 * Allow numbers like `1.` in math mode (NetBSD#50). ## 0.5.0.4 * Add built-in identifiers for standard types (NetBSD#21): array, bool, content, int, float, regex, length, alignment, color, symbol, string. * Adjust emphasis parser for CJK characters (NetBSD#49). Typst documentation says that `*` strong emphasis "only works at word boundaries." However, in typst/typst#2648 this was changed for CJK.
## 0.6.1 * Fix precedence for functions (#55). `1(x)` and `!(x)` should not be parsed as functions. Note that we still don't match typst's behavior for `f_"!"(x)`. For us this works just like `f_!(x)`, but for typst we get a function in the subscript for the former but not the latter. Fixing this would require some changes in the types. * Define sys.version and sys.inputs.typst-hs-version (#56). The former is set to the version of typst we are trying to implement. The latter is a stringified version number from typst-hs. This will allow typst programs to tell when they're running on typst-hs (or pandoc), and react accordingly. * Rename stBeforeSpace -> stSpaceBefore to avoid confusion. * Fix precedence issues in math parsing (#54). Increased precedence of ! (factorial). `_` or `^` should eagerly gobble a grouped argument (`c_(a)`). * Minimal support for `context` (#53). Parse `context` keyword. New Context constructor in Expr [API change]. Evaluate this by just evaluating the expression, for now. Note that we don't support the features (like location or numbering) that context is used to affect anyway, so this change probably won't be enough for meaningful support. But it might prevent some documents from issuing errors. * Arguments at method. * Array windows, reduce, to-dict methods. * Allow parentheses in import. * Make standard module available under std (typst 0.12). * Add over/underparen, over/undershell in math module. * Add stretch function. * Add skew. * Depend on typst-symbols 0.1.7 and start to target typst 0.12. * Reset indentation requirements inside `[]` content block. e.g. ``` / B: #block[ - a ] ``` We don't need indentation inside the block content. ## 0.6 * Recognize figure.caption function (#52). * Allow defined identifiers to override math defaults (#51). Previously we evaluated all math in a special environment that shadowed any user-defined functions with the same-named functions from the math or sym modules. This change gives user-defined identifiers priority over the math defaults, allowing things like `bb` to be overridden. * Typst.Types: EvalState now has two new fields, `evalMathIdentifiers` and `evalStandardIdentifiers`. `evalIdentifiers` is now just for user-defined identifiers. [API change] * Don't implicitly load sys module for math. ## 0.5.0.5 * Allow numbers like `1.` in math mode (#50). ## 0.5.0.4 * Add built-in identifiers for standard types (#21): array, bool, content, int, float, regex, length, alignment, color, symbol, string. * Adjust emphasis parser for CJK characters (#49). Typst documentation says that `*` strong emphasis "only works at word boundaries." However, in typst/typst#2648 this was changed for CJK.
5 participants
Chinese and Japanese do not use spaces in normal texts. So currently you always need to use function instead of markups for emphasis, which is annoying. This PR fixes that.
Related: #635 (comment)