Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PDF: Bold MathML not rendering #910

Open
opoudjis opened this issue Jan 21, 2023 · 38 comments
Open

PDF: Bold MathML not rendering #910

opoudjis opened this issue Jan 21, 2023 · 38 comments
Assignees
Labels

Comments

@opoudjis
Copy link
Contributor

Follow on from #876

I have introduced code to insert <mstyle fontweight="bold"> into MathML if its ancestor is <strong>. However, I have discovered that style directives to render MathML in boldface are being ignored in PDF, whether they are the older <mstyle fontweight="bold">, or the newer <mstyle style="font-weight:bold">, or the native <mstyle mathvariant="bold">.

So the following snippet renders the b in boldface (non-italic) as expected in HTML and DOC, but as plain style in PDF:

<stem type="MathML">
  <math xmlns="http://www.w3.org/1998/Math/MathML">
    <mi>a</mi> 
    <mstyle mathvariant="bold">
      <mi>b</mi>
    </mstyle>
  </math><!-- a b -->
  <asciimath>a bb(b)</asciimath>
</stem>

And it renders the same whether the mstyle has mathvariant="bold", style="font-weight:bold", or fontweight="bold".

@opoudjis opoudjis added the bug label Jan 21, 2023
@github-project-automation github-project-automation bot moved this to 🆕 New in Metanorma Jan 21, 2023
@Intelligent2013 Intelligent2013 moved this from 🆕 New to 🏗 In progress in Metanorma Jan 21, 2023
@Intelligent2013
Copy link
Contributor

Intelligent2013 commented Jan 21, 2023

I can't repeat. This XML:

<p id="_test">
Bold math test: 
<stem type="MathML">
  <math xmlns="http://www.w3.org/1998/Math/MathML">
    <mi>a</mi> 
    <mstyle mathvariant="bold">
      <mi>b</mi>
    </mstyle>
  </math><!-- a b -->
  <asciimath>a bb(b)</asciimath>
</stem> 
</p>

renders so:
image

@opoudjis could you attach the full xml fragment?

@Intelligent2013
Copy link
Contributor

I've test for different styles mathvariant="bold", style="font-weight:bold", or fontweight="bold", and only mathvariant="bold" renders ok in PDF:
image

@opoudjis
Copy link
Contributor Author

opoudjis commented Feb 1, 2023

The mstyle is at the root of the MathML, and is still failing in PDF with mathvariant. Attached.

Αρχειοθήκη 2.zip

@Intelligent2013
Copy link
Contributor

In my environment PDF looks so:
image

vs. from Αρχειοθήκη 2.zip:
image

I don't figure out the reason. I'll investigate.

@ronaldtse ronaldtse changed the title Bold MathML not rendering PDF: Bold MathML not rendering Feb 2, 2023
@Intelligent2013
Copy link
Contributor

The document from mn-samples-iso repo https://github.com/metanorma/mn-samples-iso/blob/main/sources/directives/part2/sections/28-figures.adoc contains similar math in bold:

| Position vector | stem:[bb(r)]
| Displacement | stem:[bb(Delta r)]
<tr><td valign="top" align="left">Position vector</td>
<td valign="top" align="left"><stem type="MathML"><math xmlns="http://www.w3.org/1998/Math/MathML"><mstyle mathvariant="bold"><mi>r</mi></mstyle></math><asciimath>bb(r)</asciimath></stem></td>
</tr><tr><td valign="top" align="left">Displacement</td>
<td valign="top" align="left"><stem type="MathML"><math xmlns="http://www.w3.org/1998/Math/MathML"><mstyle mathvariant="bold"><mrow><mo>Δ</mo><mi>r</mi></mrow></mstyle></math><asciimath>bb(Delta r)</asciimath></stem></td>
</tr>

and math in PDF rendered in bold on Win, Ubuntu and Mac (metanorma/mn-native-pdf#488), Table 10 on the page 53:
Windows (https://github.com/metanorma/mn-native-pdf/actions/runs/4073826433):
image

Ubuntu (https://github.com/metanorma/mn-native-pdf/actions/runs/4073826407):
image

MacOS (https://github.com/metanorma/mn-native-pdf/actions/runs/4073826414):
image

Will investigate further...

@Intelligent2013
Copy link
Contributor

I assume that wrong (old?) Cambria font without Math sub-font is using.

@opoudjis could you help make some actions?

  • re-generate PDF again
  • immediately after that, go to the folder /Users/<user name>/.metanorma/fonts/ and find the file pdf_fonts_config.xml.out.
  • attach it here (or via Skype)
  • in the file pdf_fonts_config.xml.out find two elements font with <font-triplet name="Cambria Math" style="normal" weight="normal"/> and <font-triplet name="Cambria Math" style="normal" weight="bold"/> inside it, like this:
<font embed-url="file:/C:/Windows/FONTS/cambria.ttc" kerning="yes" mn_default="false" sub-font="Cambria Math">
	<alternate embed-url="cambria.ttf"/>
	<font-triplet name="Cambria Math" style="normal" weight="normal"/>
</font>
<font embed-url="file:/C:/Windows/FONTS/cambria.ttc" kerning="yes" mn_default="false" simulate-style="true" sub-font="Cambria Math">
	<alternate embed-url="cambria.ttf"/>
	<font-triplet name="Cambria Math" style="normal" weight="bold"/>
</font>
  • attach the font files specified in the attribute embed-url.

Thanks.

@Intelligent2013
Copy link
Contributor

@opoudjis thanks.

From pdf_fonts_config.xml.out - Cambria.ttf is using for Cambria Math:

<font embed-url="file:/Library/Fonts/Microsoft/Cambria.ttf" kerning="yes" mn_default="false">
	<alternate embed-url="cambria.ttf"/>
	<font-triplet name="Cambria" style="normal" weight="normal"/>
</font>
<font embed-url="file:/Library/Fonts/Microsoft/Cambria.ttf" kerning="yes" mn_default="false">
	<alternate embed-url="cambria.ttf"/>
	<font-triplet name="Cambria Math" style="normal" weight="normal"/>
</font>
<font embed-url="file:/Library/Fonts/Microsoft/Cambria.ttf" kerning="yes" mn_default="false">
	<alternate embed-url="cambria.ttf"/>
	<font-triplet name="Cambria Math" style="normal" weight="bold"/>
</font>

But on the disk there is the font Cambria Math in Cambria Math.ttf.
Also, missing simulate-style="true" for <font-triplet name="Cambria Math" style="normal" weight="bold"/>.
Will fix it.

@Intelligent2013
Copy link
Contributor

I'm trying to generate PDF for a.doc on MacOS and get the error:

bundle exec metanorma -t iso -x pdf a.adoc:

Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
bundler: failed to load command: metanorma (/usr/local/lib/ruby/gems/3.2.0/bin/metanorma)
/usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/commands/config.rb:57:in `block in load_configs': undefined method `exists?' for File:Class (NoMethodError)

            next unless File.exists?(config_path)
                            ^^^^^^^^
Did you mean?  exist?
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/commands/config.rb:56:in `each'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/commands/config.rb:56:in `load_configs'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/thor_with_config.rb:14:in `options'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-cli-1.6.12/lib/metanorma/cli/command.rb:46:in `compile'
...

ruby -v:

ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-darwin20]

Will try to re-install metanorma.

@Intelligent2013
Copy link
Contributor

File.exists? were removed from Ruby 3.2.0.

@Intelligent2013
Copy link
Contributor

Issue with File.exists? fixed by adding this line into Gemfile:
gem "metanorma-cli", git: "https://github.com/metanorma/metanorma-cli", branch: "main"

New error occurs:

Fatal Error: Refinement#include has been removed
bundler: failed to load command: metanorma (/usr/local/lib/ruby/gems/3.2.0/bin/metanorma)
/usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:176:in `include': Refinement#include has been removed (TypeError)

    include ::RDF::Term
            ^^^^^^^^^^^
	from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:176:in `block in <module:Refinements>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:174:in `refine'
	from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:174:in `<module:Refinements>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3/refinements.rb:2:in `<top (required)>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3.rb:26:in `require'
	from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3.rb:26:in `<module:N3>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3.rb:21:in `<module:RDF>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/rdf-n3-3.1.2/lib/rdf/n3.rb:3:in `<top (required)>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/linkeddata-3.1.6/lib/linkeddata.rb:26:in `require'
	from /usr/local/lib/ruby/gems/3.2.0/gems/linkeddata-3.1.6/lib/linkeddata.rb:26:in `<module:LinkedData>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/linkeddata-3.1.6/lib/linkeddata.rb:1:in `<top (required)>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-w3c-1.14.0/lib/relaton_w3c/data_fetcher.rb:2:in `require'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-w3c-1.14.0/lib/relaton_w3c/data_fetcher.rb:2:in `<top (required)>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-w3c-1.14.0/lib/relaton_w3c.rb:8:in `require'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-w3c-1.14.0/lib/relaton_w3c.rb:8:in `<top (required)>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:28:in `require'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:28:in `block in register_gems'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:27:in `each'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:27:in `register_gems'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/registry.rb:21:in `initialize'
	from /usr/local/Cellar/ruby/3.2.0/lib/ruby/3.2.0/singleton.rb:127:in `new'
	from /usr/local/Cellar/ruby/3.2.0/lib/ruby/3.2.0/singleton.rb:127:in `block in instance'
	from /usr/local/Cellar/ruby/3.2.0/lib/ruby/3.2.0/singleton.rb:125:in `synchronize'
	from /usr/local/Cellar/ruby/3.2.0/lib/ruby/3.2.0/singleton.rb:125:in `instance'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/db.rb:6:in `initialize'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/db.rb:530:in `new'
	from /usr/local/lib/ruby/gems/3.2.0/gems/relaton-1.14.2/lib/relaton/db.rb:530:in `init_bib_caches'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/ref_sect.rb:166:in `init_bib_caches'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/base.rb:119:in `init_biblio'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/base.rb:40:in `init'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-iso-2.3.3/lib/metanorma/iso/base.rb:54:in `init'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/base.rb:140:in `document1'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/base.rb:130:in `document'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-standoc-2.3.7/lib/metanorma/standoc/utils.rb:13:in `convert'
	from /usr/local/lib/ruby/gems/3.2.0/gems/asciidoctor-2.0.18/lib/asciidoctor/document.rb:956:in `convert'
	from /usr/local/lib/ruby/gems/3.2.0/gems/asciidoctor-2.0.18/lib/asciidoctor/convert.rb:120:in `convert'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/input/asciidoc.rb:18:in `process'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/processor.rb:21:in `input_to_isodoc'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/compile.rb:59:in `process_input_adoc'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/compile.rb:42:in `process_input'
	from /usr/local/lib/ruby/gems/3.2.0/gems/metanorma-1.5.5/lib/metanorma/compile.rb:33:in `compile'

@opoudjis
Copy link
Contributor Author

opoudjis commented Feb 3, 2023

bundle info rdf is ok, but let me upgrade to Ruby 3.2.0....

... replicated error. Aha.

Please use Ruby 3.1.3 until we've worked this out.

relaton/relaton-w3c#34

Intelligent2013 added a commit to metanorma/mn2pdf-ruby that referenced this issue Feb 3, 2023
Intelligent2013 added a commit to metanorma/mn2pdf-ruby that referenced this issue Feb 3, 2023
@Intelligent2013
Copy link
Contributor

Please use Ruby 3.1.3 until we've worked this out.

@opoudjis thank you, it's working.

I've repeated the wrong PDF rendering with non-bold math on MacOS.

I've installed these fonts

  • Cambria.ttf
  • Cambria Bold.ttf
  • Cambria Math.ttf
    on clear MacOS installation.

Fontist manifest looks correctly:

---
Cambria:
  Bold:
    full_name: Cambria Bold
    paths:
    - "/Library/Fonts/Cambria Bold.ttf"
  Regular:
    full_name: Cambria
    paths:
    - "/Library/Fonts/Cambria.ttf"
Cambria Math:
  Regular:
    full_name: Cambria Math
    paths:
    - "/Library/Fonts/Cambria Math.ttf"
...

In the FOP font config (parsed from fontist manifest):

<fonts>
	<font embed-url="file:/Library/Fonts/Cambria.ttf" kerning="yes" mn_default="false" sub-font="Cambria">
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria" style="normal" weight="normal"/>
	</font>
	<font embed-url="file:/Library/Fonts/Cambria%20Math.ttf" kerning="yes" mn_default="false" sub-font="Cambria Math">
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria Math" style="normal" weight="normal"/>
	</font>
	<font embed-url="file:/Library/Fonts/Cambria.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria Math" style="normal" weight="bold"/>
	</font>
	<font embed-url="file:/Library/Fonts/Cambria%20Bold.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="Cambria Bold.ttf"/>
		<font-triplet name="Cambria" style="normal" weight="bold"/>
	</font>

there are a few wrong items:

  • for Cambria.ttf specified sub-font="Cambria", but in .ttf there aren't sub-fonts, only in .ttc
  • for Cambria.ttf - <font-triplet name="Cambria Math" style="normal" weight="bold"/>
  • there isn't <font-triplet name="Cambria Math" style="normal" weight="bold"/> for file:/Library/Fonts/Cambria%20Math.ttf

There is a differences between my FOP font config (left panel) and provided by @opoudjis (right panel):
image

  • there is a redundant sub-font on the left
  • there isn't Cambria Math on the right at all

I don't figure out why Cambria Math is missing.

At this moment I see two issues:

  • wrong fontist manifest parsing by mn2pdf
  • wrong fontist manifest generation on @opoudjis machine (don't sure).

@opoudjis to make sure that fontist manifest is correctly, could you make some actions. please?
Fontist manifest .yml file generates as temporary file, therefore it deletes by Ruby garbage collector. To catch the fontist manifest file I've updated mn2pdf-ruby in the branch fontist_catch.
Add this line into Gemfile:
gem "mn2pdf", git: "https://github.com/metanorma/mn2pdf-ruby", branch: "fontist_catch"
Execute bundle update and then run PDF generation.

In the log, before the line java -Xss5m -Xmx2048m you will see the line like C:/Users/TestUser/AppData/Local/Temp/fontist_locations20230203-7164-rldc4h.yml.copy.yml:

C:/Users/TestUser/AppData/Local/Temp/fontist_locations20230203-7164-rldc4h.yml.copy.yml
java -Xss5m -Xmx2048m -jar C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/../bin/mn2pdf.jar --xml-file "D:/Work/Metanorma/repositories/mn-samples-iso/sources/math_bold/a.presentati
on.xml" --xsl-file "C:/tools/ruby31/lib/ruby/gems/3.1.0/gems/metanorma-iso-2.3.3/lib/isodoc/iso/iso.international-standard.xsl" --pdf-file "D:/Work/Metanorma/repositories/mn-samples-iso/sources/math_bold/a.pdf"
--syntax-highlight  --font-manifest "C:/Users/TestUser/AppData/Local/Temp/fontist_locations20230203-7164-rldc4h.yml"
C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/mn2pdf.rb:83:in `mn2pdf'
C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/mn2pdf.rb:58:in `block in convert'
C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/mn2pdf.rb:104:in `block in dump_fontist_manifest_locations'
...
[mn2pdf] Fatal:  Error: Unable to access jarfile C:/tools/ruby31/lib/ruby/gems/3.1.0/bundler/gems/mn2pdf-ruby-fc9a124159b9/lib/../bin/mn2pdf.jar

Please attach this .yml.copy.yml.

The process ends with error due missing mn2pdf.jar, so ignore it. I need just fontist manifest yml.

Thanks.

@Intelligent2013 Intelligent2013 moved this from 🏗 In progress to 🌋 Urgent in Metanorma Mar 10, 2023
@Intelligent2013
Copy link
Contributor

I'll make one more attempts to fix this issue...

Intelligent2013 added a commit to metanorma/mn2pdf that referenced this issue Mar 12, 2023
Intelligent2013 added a commit to metanorma/mn2pdf that referenced this issue Mar 12, 2023
@Intelligent2013
Copy link
Contributor

After update (mn2pdf v.1.64) Apache FOP config looks correct:

<fonts>
	<font embed-url="file:/Library/Fonts/Cambria.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria" style="normal" weight="normal"/>
	</font>
	<font embed-url="file:/Library/Fonts/Cambria%20Math.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="Cambria Math.ttf"/>
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria Math" style="normal" weight="normal"/>
	</font>
	<font embed-url="file:/Library/Fonts/Cambria Math.ttf" kerning="yes" mn_default="false" simulate-style="true">
		<alternate embed-url="Cambria Math.ttf" simulate-style="true"/>
		<alternate embed-url="cambria.ttf" simulate-style="true"/>
		<font-triplet name="Cambria Math" style="normal" weight="bold"/>
	</font>
	<font embed-url="file:/Library/Fonts/Cambria%20Bold.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="Cambria Bold.ttf"/>
		<font-triplet name="Cambria" style="normal" weight="bold"/>
	</font>

but PDF still renders in the normal:
image

To do:

  • check missing %20 in Cambria Math.ttf
  • check simulate-style="true" is workable or not for Cambria Math.ttf

Intelligent2013 added a commit to metanorma/mn2pdf-ruby that referenced this issue Mar 13, 2023
@Intelligent2013
Copy link
Contributor

Intelligent2013 commented Mar 13, 2023

Looks like this
image
renders in non Cambria Math Bold at all.

Cambria Math bold should looks so:
image

I'll debug jEuclid rendering.

@Intelligent2013
Copy link
Contributor

I've spent a day, but didn't find the reason yet...

@opoudjis
Copy link
Contributor Author

OK, I've already completely forgotten where this came from, and I don't want to add a never ending ticket to your workload, so...

Is this a font issue, specific to Cambria Math Bold?

Is there something I can do in markup to push it along?

I'm confused that you are getting boldface and I'm not; this looks like something that can be gotten around, but it also sounds like it needs explicit invocation of the distinct Cambria Bold. Does this issue generalise to other bold math fonts? I know we've had problems with boldface maths in the past—incomplete coverage of maths symbols in fonts.

@Intelligent2013
Copy link
Contributor

@opoudjis I assume that the reason in jEuclid.
There is 'Cambria Math' only font, not 'Cambria Math Bold'.
Apache FOP can 'simulate' font's styles. I.e. if there is only normal/regular font, then Apache FOP automatically 'build' bold/italic fonts. This works ok for text.
jEuclid is using for mathml rendering, and it seems jEeuclid doesn't know anything about how to simulate the font.
I didn't have enough time to focused on this issue. As soon I've ended with JIS XSLT, then I'll switch to here again....
I think about workaround solution - create on-fly 'Cambria Math Bold' .ttf and specify it for jEuclid...

@Intelligent2013
Copy link
Contributor

After today's investigation, as I understood

  • jEuclid uses ImageGraphics2D object (org.apache.xmlgraphics.image.loader.impl.ImageGraphics2D) for drawing formulas, and then converts it into the curves, and put into PDF, I don't sure that SVG is using
  • jEuclid uses own 'simulate style' algorithm than Apache FOP:

left - jEuclid curves, right - Apache FOP text

  • Cambria Math
    image
  • STIX Two Math:
    image

Just for experiment, I've removed simulate-style="true" for STIX Two Math font, and jEuclid still renders in the bold:
image

The reason why math isn't bolded on MacOs in the investigation still.

@Intelligent2013
Copy link
Contributor

For further investigation, example XML:

    <clause id="id1" displayorder="1">
      <title depth="1">1<tab/>Math bold test</title>
      <p>Cambria math: <font font-family="Cambria Math">[a,b], <strong>[a,b]</strong>
        </font>
        <stem type="MathML" font-family="Cambria Math">
          <math xmlns="http://www.w3.org/1998/Math/MathML">
            <mstyle mathvariant="bold">
              <mfenced open="[" close="]">
                <mrow>
                  <mi>a</mi>
                  <mo>,</mo>
                  <mi>b</mi>
                </mrow>
              </mfenced>
            </mstyle>
          </math>
          <!-- [a , b] -->
          <asciimath>[a,b]</asciimath>
        </stem>
      </p>
      <p>STIX Two Math: <font font-family="STIX Two Math">[a,b], <strong>[a,b]</strong>
        </font>
        <stem type="MathML" font-family="STIX Two Math">
          <math xmlns="http://www.w3.org/1998/Math/MathML">
            <mstyle mathvariant="bold">
              <mfenced open="[" close="]">
                <mrow>
                  <mi>a</mi>
                  <mo>,</mo>
                  <mi>b</mi>
                </mrow>
              </mfenced>
            </mstyle>
          </math>
          <!-- [a , b] -->
          <asciimath>[a,b]</asciimath>
        </stem>
      </p>
    </clause>

XSLT updated for new tags (for testing only):

 <xsl:template match="*[local-name() = 'font']">
    <fo:inline font-family="{@font-family}"><xsl:apply-templates /></fo:inline>
 </xsl:template>

<xsl:template match="mathml:math">
...
<fo:inline xsl:use-attribute-sets="mathml-style">
      <xsl:copy-of select="ancestor::*[local-name() = 'stem']/@font-family"/>
...

Fontist manifest fragment (Cambria fonts from Mac machine specially for testing):

---
Cambria:
  Regular:
    full_name: Cambria
    paths:
    -"D:/Work/Metanorma/Mac_fonts/Cambria.ttf"
  Bold:
    full_name: Cambria Bold
    paths:
    - "D:/Work/Metanorma/Mac_fonts/Cambria Bold.ttf"
...
Cambria Math:
  Regular:
    full_name: Cambria Math
    paths:
    - "D:/Work/Metanorma/Mac_fonts/Cambria Math.ttf"
STIX Two Math:
  Regular:
    full_name: STIX Two Math Regular
    paths:
    - C:/Users/TestUser/.fontist/fonts/STIXTwoMath-Regular.otf

Currently rendering on Win10:
image

Note: first [a,b] is normal text, 2nd is bold text, 3rd is jEuclid graphics.

I'll try on Mac tomorrow, something wrong with VirtualBox.

@Intelligent2013
Copy link
Contributor

Intelligent2013 commented Apr 20, 2023

Rendering on Mac:
image

jEuclid renders bold in STIX Two Math, but doesn't in Cambria Math. Very uncertain result...

Further actions:

  • remove space in the filename Cambria Math.ttf on the disk and in manifest (may be jEuclid doesn't understand spaces in the filename?)
  • try cambria.ttc (contains both Cambria and Cambria Math) from Windows.

@Intelligent2013
Copy link
Contributor

  • remove space in the filename Cambria Math.ttf on the disk and in manifest (may be jEuclid doesn't understand spaces in the filename?)
  • try cambria.ttc (contains both Cambria and Cambria Math) from Windows.

The same result - no bold.

@Intelligent2013
Copy link
Contributor

JEuclid MathViewer with fontSerif attribute:
image

on Windows :

  • with mathvariant="bold"
    image
  • without mathvariant="bold"
    image

Note: bold looks different than as it renders in PDF.

on Mac

with and without mathvariant="bold" looks identically
image

@Intelligent2013
Copy link
Contributor

This code from Mo.java has affect on the bold style for brackets:

 info.setGraphicsObject(new TextObject(t, this.getLspaceAsFloat(now)
                + textLayoutInfo.getOffset() * calcScaleX, calcBaselineShift,
                AffineTransform.getScaleInstance(calcScaleX, calcScaleY), //calcScaleX, calcScaleY
                (Color) now.getParameter(Parameter.MATHCOLOR)));

For calcScaleX=1.0f and calcScaleY=1.0f, PDF renders (on Windows):
image
For calcScaleX=1.5f and calcScaleY=1.0f, PDF renders (on Windows):
image

For both case in Mac:
image

The objects textLayout is identical on both platforms:

textLayout=sun.font.StandardTextSource@671c4166[start:0, len:1, cstart:0, clen:1, chars:"5b", level:0, flags:0, font:java.awt.Font[family=Cambria Math,name=Cambria Math,style=bold,size=11], frc:java.awt.font.FontRenderContext@78f755b2, cm:sun.font.CoreMetrics@41183d00]
textLayout=sun.font.StandardTextSource@228cea97[start:0, len:1, cstart:0, clen:1, chars:"5b", level:0, flags:0, font:java.awt.Font[family=Cambria Math,name=Cambria Math,style=bold,size=11], frc:java.awt.font.FontRenderContext@6e20c58e, cm:sun.font.CoreMetrics@41273c00]

@Intelligent2013
Copy link
Contributor

It's the Java issue.
I've developed a small program to check the math fonts rendering in Java GUI (https://github.com/Intelligent2013/mathfontstest).

  • 1st and 3rd lines in normal
  • 2nd and 4th lines in bold
        JLabel label1 = new JLabel("Cambria Math: [a, b]");
        label1.setBounds(50, 50, 400, 30);
        label1.setFont(new Font("Cambria Math", Font.PLAIN, 30));

        JLabel label2 = new JLabel("Cambria Math: [a, b]");
        label2.setBounds(50, 100, 400, 30);
        label2.setFont(new Font("Cambria Math", Font.BOLD, 30));

        JLabel label3 = new JLabel("STIX Two Math: [a, b]");
        label3.setBounds(50, 150, 400, 30);
        label3.setFont(new Font("STIX Two Math", Font.PLAIN, 30));

        JLabel label4 = new JLabel("STIX Two Math: [a, b]");
        label4.setBounds(50, 200, 400, 30);
        label4.setFont(new Font("STIX Two Math", Font.BOLD, 30));

On Windows:
image

On Mac - no differences between 1st and 2nd lines:
image

And I don't figure out why Cambria Math looks thickness on Mac than on Windows.

@Intelligent2013
Copy link
Contributor

On Ubuntu:
image

@Intelligent2013
Copy link
Contributor

There are the similar issue on the openjdk bug tracker:

@Intelligent2013
Copy link
Contributor

Previously, I've installed the font Cambria Math.ttf into the folder /Library/Fonts on my VB MacOs (just to repeat the environment from the original issue MacOS machine).
But now I've moved it into ~/.fontist/fonts/... and now it renders ok in the test application MathFontsTest:
image

Note: now, 'normal' renders as normal font, not 'bold' as in #910 (comment)

I'll test the PDF rendering and will think about one more workaround solution, specially for MacOs.

@Intelligent2013
Copy link
Contributor

Case for ~/.fontist/fonts/Cambria Math.ttf - PDF renders OK:
image

So, workaround solution for mn2pdf:

  • if OS is MacOS
  • if font file path doesn't contain .fontist (i.e. isn't installed by fontist), then
    • copy font file into tmp folder (one folder for all fonts)
    • change font path to the tmp folder font file
    • generate PDF...
    • flush font tmp folder

@Intelligent2013
Copy link
Contributor

Current FOP config:

<fonts>
	<font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/3e1e15c6-5e2b-40e8-93eb-48996f722bfc/Cambria.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria" style="normal" weight="normal"/>
	</font>
	<font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/3e1e15c6-5e2b-40e8-93eb-48996f722bfc/Cambria%20Math.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="Cambria Math.ttf"/>
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria Math" style="normal" weight="normal"/>
	</font>
	<font embed-url="file:/Library/Fonts/Cambria Math.ttf" kerning="yes" mn_default="false" simulate-style="true">
		<alternate embed-url="Cambria Math.ttf" simulate-style="true"/>
		<alternate embed-url="cambria.ttf" simulate-style="true"/>
		<font-triplet name="Cambria Math" style="normal" weight="bold"/>
	</font>
	<font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/3e1e15c6-5e2b-40e8-93eb-48996f722bfc/Cambria%20Bold.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="Cambria Bold.ttf"/>
		<font-triplet name="Cambria" style="normal" weight="bold"/>
	</font>

There is file:/Library/Fonts/Cambria Math.ttf still.
To do: for non-fontist manifest entries apply the same actions.

@ronaldtse
Copy link
Contributor

@Intelligent2013 is there something that Fontist can do here? I don't like the idea of mn2pdf moving fonts around.

@Intelligent2013
Copy link
Contributor

For FOP font config:

<fonts>
	<font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/8a88be7d-05a5-4ed7-9b34-f9184598daef/Cambria.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria" style="normal" weight="normal"/>
	</font>
	<font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/8a88be7d-05a5-4ed7-9b34-f9184598daef/Cambria%20Math.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="Cambria Math.ttf"/>
		<alternate embed-url="cambria.ttf"/>
		<font-triplet name="Cambria Math" style="normal" weight="normal"/>
	</font>
	<font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/8a88be7d-05a5-4ed7-9b34-f9184598daef/Cambria%20Math.ttf" kerning="yes" mn_default="false" simulate-style="true">
		<alternate embed-url="Cambria Math.ttf" simulate-style="true"/>
		<alternate embed-url="cambria.ttf" simulate-style="true"/>
		<font-triplet name="Cambria Math" style="normal" weight="bold"/>
	</font>
	<font embed-url="file:/var/folders/vn/4jyz_ysn61lbh37__jrqsx1c0000gn/T/8a88be7d-05a5-4ed7-9b34-f9184598daef/Cambria%20Bold.ttf" kerning="yes" mn_default="false">
		<alternate embed-url="Cambria Bold.ttf"/>
		<font-triplet name="Cambria" style="normal" weight="bold"/>
	</font>

PDF again wrong:
image

Looks like JEuclid uses the system information about registered fonts, and therefore uses the font from /Library/Fonts/.
I'll try to unregister font for Java...

@Intelligent2013
Copy link
Contributor

I can't find how to unregister font for JVM...

One more experiment without font removing: as soon I disable the font 'Cambria Math' in FontBox manually, PDF renders ok:
image

I need timeout...

@Intelligent2013
Copy link
Contributor

@Intelligent2013 is there something that Fontist can do here? I don't like the idea of mn2pdf moving fonts around.

@ronaldtse we need to isolate system fonts from Java somehow, but I don't know how yet.

@Intelligent2013
Copy link
Contributor

@Intelligent2013
Copy link
Contributor

I have one idea:

  • copy system font to the temp folder
  • register the font from the temp folder with a new name, for instance, Cambria Math Metanorma
  • replace Cambria Math to Cambria Math Metanorma in the XSL-FO

Java doesn't allow to change font name directly.
Currently, I've found the method how to change the font name in the Java Font object (via reflection feature):
image

But GraphicsEnvironment ge.registerFont(ttfFont) doesn't register this updated font, because familyName and fullName still have value Cambria Math (without Metanorma suffix):

Locale l = getSystemStartupLocale();
String familyName = font.getFamily(l).toLowerCase();
String fullName = font.getFontName(l).toLowerCase();

The methods getFontName and getFamilyName in the class Font2D just return fullName and familyName:

public String getFontName(Locale l) {
        return fullName;
}

public String getFamilyName(Locale l) {
        return familyName;
 }

@Intelligent2013
Copy link
Contributor

Intelligent2013 commented Jun 22, 2023

Actually, the right class is TrueTypeFont (Font2D is abstract class):

    @Override
    public String getFontName(Locale locale) {
        if (locale == null) {
            return fullName;
        } else if (locale.equals(nameLocale) && localeFullName != null) {
            return localeFullName;
        } else {
            short localeID = getLCIDFromLocale(locale);
            String name = lookupName(localeID, FULL_NAME_ID);
            if (name == null) {
                return fullName;
            } else {
                return name;
            }
        }
    }

The font's name depends on locale and stores very deep into the byte array:

public static final int nameTag = 0x6E616D65; // 'name'
...
   /* Return the requested name in the requested locale, for the
     * MS platform ID. If the requested locale isn't found, return US
     * English, if that isn't found, return null and let the caller
     * figure out how to handle that.
     */
    protected String lookupName(short findLocaleID, int findNameID) {
        String foundName = null;
        byte[] name = new byte[1024];

        ByteBuffer buffer = getTableBuffer(nameTag);
        if (buffer != null) {
            ShortBuffer sbuffer = buffer.asShortBuffer();
            sbuffer.get(); // format - not needed.
            short numRecords = sbuffer.get();

            /* The name table uses unsigned shorts. Many of these
             * are known small values that fit in a short.
             * The values that are sizes or offsets into the table could be
             * greater than 32767, so read and store those as ints
             */
            int stringPtr = ((int) sbuffer.get()) & 0xffff;

            for (int i=0; i<numRecords; i++) {
                short platformID = sbuffer.get();
                if (platformID != MS_PLATFORM_ID) {
                    sbuffer.position(sbuffer.position()+5);
                    continue; // skip over this record.
                }
                short encodingID = sbuffer.get();
                short langID     = sbuffer.get();
                short nameID     = sbuffer.get();
                int   nameLen    = ((int) sbuffer.get()) & 0xffff;
                int   namePtr    = (((int) sbuffer.get()) & 0xffff) + stringPtr;
                if (nameID == findNameID &&
                    ((foundName == null && langID == ENGLISH_LOCALE_ID)
                     || langID == findLocaleID)) {
                    buffer.position(namePtr);
                    buffer.get(name, 0, nameLen);
                    foundName = makeString(name, nameLen, encodingID);
                    if (langID == findLocaleID) {
                        return foundName;
                    }
                }
            }
        }
        return foundName;
    }

So, it's very hard to change it.
I assume that would be easier to change the locale before font registering to en_US and restore it to the previous value after.
In this case the values from localeFamilyName and localeFontName will be used.

@Intelligent2013
Copy link
Contributor

I assume that would be easier to change the locale before font registering to en_US and restore it to the previous value after.
In this case the values from localeFamilyName and localeFontName will be used.

No, this approach isn't working, because 'locale' determines via the system properties:

  • from TrueTypeFont:
nameLocale = sun.awt.SunToolkit.getStartupLocale();

...

} else if (locale.equals(nameLocale) && localeFullName != null) {
            return localeFullName;

where locale determines in SunFontManager:

                String fileEncoding = System.getProperty("file.encoding", "");
                String sysEncoding = System.getProperty("sun.jnu.encoding");
                if (sysEncoding != null && !sysEncoding.equals(fileEncoding)) {
                    return Locale.ROOT;
                }

                String language = System.getProperty("user.language", "en");
                String country  = System.getProperty("user.country","");
                String variant  = System.getProperty("user.variant","");
                return new Locale(language, country, variant);
  • and nameLocale determines in SunToolkit:
    public static Locale getStartupLocale() {
        if (startupLocale == null) {
            String language, region, country, variant;
            language = AccessController.doPrivileged(
                            new GetPropertyAction("user.language", "en"));
            // for compatibility, check for old user.region property
            region = AccessController.doPrivileged(
                            new GetPropertyAction("user.region"));
            if (region != null) {
                // region can be of form country, country_variant, or _variant
                int i = region.indexOf('_');
                if (i >= 0) {
                    country = region.substring(0, i);
                    variant = region.substring(i + 1);
                } else {
                    country = region;
                    variant = "";
                }
            } else {
                country = AccessController.doPrivileged(
                                new GetPropertyAction("user.country", ""));
                variant = AccessController.doPrivileged(
                                new GetPropertyAction("user.variant", ""));
            }
            startupLocale = new Locale(language, country, variant);
        }
        return startupLocale;
    }

Need find the another way for the workaround solution.

Intelligent2013 added a commit to metanorma/mn2pdf that referenced this issue Jul 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: 🌋 Urgent
Development

No branches or pull requests

3 participants