Code

Introduce mBart#29

Code

Open

kauterry wants to merge 4 commits intomainfacebookresearch/fairseq2:mainfrom mbart_tokenizerfacebookresearch/fairseq2:mbart_tokenizerCopy head branch name to clipboard

+562

Commits

Commits on Sep 9, 2023

• Load a base mbart model and implement its text tokenizer.

  kauterry

  committed
  023e30c

  Copy full SHA for 023e30c

Commits on Sep 11, 2023

• Fixing bug in lang control symbol in prefix, suffix tokens.

  kauterry

  committed
  0eb2328

  Copy full SHA for 0eb2328

• Changing lang tag, removing norm_order, setting to pre-LN.

  kauterry

  committed
  c52ce3a

  Copy full SHA for c52ce3a

Commits on Sep 15, 2023

• Embedding special token reordering to align with fairseq.

  kauterry

  committed
  c7c7e91

  Copy full SHA for c7c7e91