Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ttslearnにおける転移学習について #42

Open
mule-engineer13 opened this issue Aug 29, 2022 · 2 comments
Open

ttslearnにおける転移学習について #42

mule-engineer13 opened this issue Aug 29, 2022 · 2 comments

Comments

@mule-engineer13
Copy link

初めまして。mule-engineer13と申します。
『Pythonで学ぶ音声合成』を参考に音声合成の学習を進めております。
1点質問をさせていただきます。

ttslearnのtacotron2とwavenetボコーダについて、転移学習を行うことは可能でしょうか?
可能であれば、方法を伺えますと幸いです。
何卒よろしくお願いいたします。

@r9y9
Copy link
Owner

r9y9 commented Sep 4, 2022

回答が遅くなりすみません。可能です。リポジトリにソースコードが含まれていますので、下記のコードを参考にしていただければと思います。

レシピ:

if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
echo "stage 3: Training Tacotron"
if [ ${finetuning} = "true" ] && [ -z ${pretrained_acoustic_checkpoint} ]; then
pretrained_acoustic_checkpoint=$PWD/../../jsut/tacotron2_pwg/exp/jsut_sr${sample_rate}/${acoustic_model}/${acoustic_eval_checkpoint}
if [ ! -e $pretrained_acoustic_checkpoint ]; then
echo "Please first train a acoustic model for JSUT corpus!"
echo "Expected model path: $pretrained_acoustic_checkpoint"
exit 1
fi
fi
xrun python train_tacotron.py model=$acoustic_model tqdm=$tqdm \
data.train.utt_list=data/train.list \
data.train.in_dir=$dump_norm_dir/$train_set/in_tacotron/ \
data.train.out_dir=$dump_norm_dir/$train_set/out_tacotron/ \
data.dev.utt_list=data/dev.list \
data.dev.in_dir=$dump_norm_dir/$dev_set/in_tacotron/ \
data.dev.out_dir=$dump_norm_dir/$dev_set/out_tacotron/ \
train.out_dir=$expdir/${acoustic_model} \
train.log_dir=tensorboard/${expname}_${acoustic_model} \
train.max_train_steps=$tacotron_train_max_train_steps \
data.batch_size=$tacotron_data_batch_size \
cudnn.benchmark=$cudnn_benchmark cudnn.deterministic=$cudnn_deterministic \
train.pretrained.checkpoint=$pretrained_acoustic_checkpoint
fi
if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then
echo "stage 4: Training Parallel WaveGAN"
if [ ${finetuning} = "true" ] && [ -z ${pretrained_vocoder_checkpoint} ]; then
voc_expdir=$PWD/../../jsut/tacotron2_pwg/exp/jsut_sr${sample_rate}/${vocoder_model}
pretrained_vocoder_checkpoint="$(ls -dt "$voc_expdir"/*.pkl | head -1 || true)"
if [ ! -e $pretrained_vocoder_checkpoint ]; then
echo "Please first train a PWG model for JSUT corpus!"
echo "Expected model path: $pretrained_vocoder_checkpoint"
exit 1
fi
extra_args="--resume $pretrained_vocoder_checkpoint"
else
extra_args=""
fi
xrun parallel-wavegan-train --config $parallel_wavegan_config \
--train-dumpdir $dump_norm_dir/$train_set/out_tacotron \
--dev-dumpdir $dump_norm_dir/$dev_set/out_tacotron/ \
--outdir $expdir/$vocoder_model $extra_args
fi

Pythonコード:

# (optional) 学習済みモデルの読み込み
# ファインチューニングしたい場合
pretrained_checkpoint = config.train.pretrained.checkpoint
if pretrained_checkpoint is not None and len(pretrained_checkpoint) > 0:
logger.info(
"Fine-tuning! Loading a checkpoint: {}".format(pretrained_checkpoint)
)
checkpoint = torch.load(pretrained_checkpoint, map_location=device)
state_dict = checkpoint["state_dict"]
model_dict = model.state_dict()
state_dict = {k: v for k, v in state_dict.items() if k in model_dict}
invalid_keys = []
for k, v in state_dict.items():
if model_dict[k].shape != v.shape:
logger.info(f"Skip loading {k}")
invalid_keys.append(k)
for k in invalid_keys:
state_dict.pop(k)
model_dict.update(state_dict)
model.load_state_dict(model_dict)

@mule-engineer13
Copy link
Author

ご回答ありがとうございます!
早速試してみます。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants