How to Fine-tune a generator? #3208
-
|
Hi. I want to use a generator without a retriever. I have short phrases (less than a complete sentence) that I need to convert to a standardized form. For this purpose, I want to fine tune a T5 model in generative QA form, wherein I ask the question, 'What is the standardized form of the given phrase?' and the answer would be the standardized form of the phrase. I could not find a way to fine tune T5 on haystack for this purpose. I believe RAG and LFQA do not suit my use case. After going through some previous discussions here, I guess Seq2SeqGenerator may be suitable, but can't figure out how I can fine tune it. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 2 replies
-
|
Hi @SigProcess , Based on your description, I think you're correct in that it's a Text2Text task. So if it's always about getting the standardized form, then perhaps the query part is not relevant. So a direct Text2Text task: And so Seq2SeqGenerator as you say is right. However, it doesn't support finetuning at the moment. It is a wrapper around HuggingFace Text2Text models. So perhaps you could look around specifically for fine-tuning those models for paraphrasing. One example. cc: @vblagoje (in case there is something incorrect in the assessment above or you have something to add) |
Beta Was this translation helpful? Give feedback.
-
|
Hi @bglearning, |
Beta Was this translation helpful? Give feedback.
-
|
Hi, can you reply to #3235 ? |
Beta Was this translation helpful? Give feedback.
Hi @SigProcess ,
Based on your description, I think you're correct in that it's a Text2Text task. So if it's always about getting the standardized form, then perhaps the query part is not relevant. So a direct Text2Text task:
non-standard_phrase->standardized-form. More specifically, sounds like a paraphrasing task.And so Seq2SeqGenerator as you say is right. However, it doesn't support finetuning at the moment. It is a wrapper around HuggingFace Text2Text models. So perhaps you could look around specifically for fine-tuning those models for paraphrasing. One example.
cc: @vblagoje (in case there is something incorrect in the assessment above or you have something to add)