QuestionAnswerGenerationPipeline -- Generative Answers?

[demongolem-biz2]

I have played around with the QuestionGenerationPipeline and it is gives me good questions.

So I quite naturally want to answer some of these questions. The immediate solution would be QuestionAnswerGenerationPipeline. However, I think that it may give extractive answers? I would like to know can one hook up the QuestionGenerator to Generative Question Answering to answer those questions?

[demongolem-biz2]

Ok so seemingly I have figured it out on my own.

The answer to my question as far as I can tell is that extractive answers are given. However, it is possibly to hook a QuestionGenerator up to a PromptTemplate where RAF is being used and we can see that generative answers are given.

[bilgeyucel]

Hi @demongolem-biz2, you can do it but you need to format the output of QuestionGenerator slightly so to connect it with PromptNode. Here's how you can do it:

from haystack.schema import Document
from haystack.nodes import QuestionGenerator
from haystack.nodes import PromptNode, PromptTemplate

text1 = "Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum and first released in 1991, Python's design philosophy emphasizes code readability with its notable use of significant whitespace."
text2 = "Princess Arya Stark is the third child and second daughter of Lord Eddard Stark and his wife, Lady Catelyn Stark. She is the sister of the incumbent Westerosi monarchs, Sansa, Queen in the North, and Brandon, King of the Andals and the First Men. After narrowly escaping the persecution of House Stark by House Lannister, Arya is trained as a Faceless Man at the House of Black and White in Braavos, using her abilities to avenge her family. Upon her return to Westeros, she exacts retribution for the Red Wedding by exterminating the Frey male line."
text3 = "Dry Cleaning are an English post-punk band who formed in South London in 2018.[3] The band is composed of vocalist Florence Shaw, guitarist Tom Dowse, bassist Lewis Maynard and drummer Nick Buxton. They are noted for their use of spoken word primarily in lieu of sung vocals, as well as their unconventional lyrics. Their musical stylings have been compared to Wire, Magazine and Joy Division.[4] The band released their debut single, 'Magic of Meghan' in 2019. Shaw wrote the song after going through a break-up and moving out of her former partner's apartment the same day that Meghan Markle and Prince Harry announced they were engaged.[5] This was followed by the release of two EPs that year: Sweet Princess in August and Boundary Road Snacks and Drinks in October. The band were included as part of the NME 100 of 2020,[6] as well as DIY magazine's Class of 2020.[7] The band signed to 4AD in late 2020 and shared a new single, 'Scratchcard Lanyard'.[8] In February 2021, the band shared details of their debut studio album, New Long Leg. They also shared the single 'Strong Feelings'.[9] The album, which was produced by John Parish, was released on 2 April 2021.[10]"

documents=[Document(text1), Document(text2), Document(text3)]

question_generator = QuestionGenerator()
generated_questions = question_generator.run(documents)

prompt_node = PromptNode(model_name_or_path="google/flan-t5-large")
answer_generator = PromptTemplate(
    prompt="Given the document {documents}, answer the questions. \n Questions:{query} \n Answer:"
)
prompt_node.run(prompt_template=answer_generator, documents=[generated_questions[0]["documents"][0]], query=generated_questions[0]["generated_questions"][0]["questions"]) 

Then PromptNode will generate answers for questions generated for text1

Also, you can use loop to run PromptNode for all documents:

for i, doc in enumerate(generated_questions[0]["documents"]):
  query = generated_questions[0]["generated_questions"][i]["questions"]
  documents = [doc]
  results = prompt_node.run(prompt_template=answer_generator, documents=documents, query=query)
  print("========\n", results)

If you want to create a custom pipeline, you can update required parts of QuestionAnswerGenerationPipeline. Let me know if you have any questions 🙌

[SiamakShams]

Even a year after your clear example Bilge, this re[;u solved a problem I've had for days.
A related question, are the ready-made pipelines like 'QuestionGenerator' available in Haystack 2?

Thanks

[demongolem-biz2]

Thanks so much. That helps give me intuition as to what is going on. Yes I could create a custom pipeline. Thus far when trying both ways, that is having the approach where the answering model is extractive versus generative, the extractive approach works pretty well out of the box and generative seems to give me wilder answers which aren't quite on point. I have a lot of work still in front of me.

…

On Wed, Oct 4, 2023 at 6:13 AM Bilge Yücel ***@***.***> wrote: Hi @demongolem-biz2 <https://github.com/demongolem-biz2>, you can do it but you need to format the output of QuestionGenerator slightly so to connect it with PromptNode. Here's how you can do it: from haystack.schema import Documentfrom haystack.nodes import QuestionGeneratorfrom haystack.nodes import PromptNode, PromptTemplate text1 = "Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum and first released in 1991, Python's design philosophy emphasizes code readability with its notable use of significant whitespace."text2 = "Princess Arya Stark is the third child and second daughter of Lord Eddard Stark and his wife, Lady Catelyn Stark. She is the sister of the incumbent Westerosi monarchs, Sansa, Queen in the North, and Brandon, King of the Andals and the First Men. After narrowly escaping the persecution of House Stark by House Lannister, Arya is trained as a Faceless Man at the House of Black and White in Braavos, using her abilities to avenge her family. Upon her return to Westeros, she exacts retribution for the Red Wedding by exterminating the Frey male line."text3 = "Dry Cleaning are an English post-punk band who formed in South London in 2018.[3] The band is composed of vocalist Florence Shaw, guitarist Tom Dowse, bassist Lewis Maynard and drummer Nick Buxton. They are noted for their use of spoken word primarily in lieu of sung vocals, as well as their unconventional lyrics. Their musical stylings have been compared to Wire, Magazine and Joy Division.[4] The band released their debut single, 'Magic of Meghan' in 2019. Shaw wrote the song after going through a break-up and moving out of her former partner's apartment the same day that Meghan Markle and Prince Harry announced they were engaged.[5] This was followed by the release of two EPs that year: Sweet Princess in August and Boundary Road Snacks and Drinks in October. The band were included as part of the NME 100 of 2020,[6] as well as DIY magazine's Class of 2020.[7] The band signed to 4AD in late 2020 and shared a new single, 'Scratchcard Lanyard'.[8] In February 2021, the band shared details of their debut studio album, New Long Leg. They also shared the single 'Strong Feelings'.[9] The album, which was produced by John Parish, was released on 2 April 2021.[10]" documents=[Document(text1), Document(text2), Document(text3)] question_generator = QuestionGenerator()generated_questions = question_generator.run(documents) prompt_node = PromptNode(model_name_or_path="google/flan-t5-large")answer_generator = PromptTemplate( prompt="Given the document {documents}, answer the questions. \n Questions:{query} \n Answer:" )prompt_node.run(prompt_template=answer_generator, documents=[generated_questions[0]["documents"][0]], query=generated_questions[0]["generated_questions"][0]["questions"]) Then PromptNode will generate answers for questions generated for text1 Also, you can use loop to run PromptNode for all documents: for i, doc in enumerate(generated_questions[0]["documents"]): query = generated_questions[0]["generated_questions"][i]["questions"] documents = [doc] results = prompt_node.run(prompt_template=answer_generator, documents=documents, query=query) print("========\n", results) If you want to create a custom pipeline, you can update required parts of QuestionAnswerGenerationPipeline. Let me know if you have any questions 🙌 — Reply to this email directly, view it on GitHub <#5930 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A45OQCLHESBT7JSD5OQS54DX5UZFTAVCNFSM6AAAAAA5MXY5U6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TCOBVGEYDC> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bilgeyucel]

@demongolem-biz2 Please update this entry with your progress and feel free to join the Haystack community on discord. People there are quite helpful and friendly 😊