[BUG]: Second Response Empty when using Grammar

[L-Wehmschulte]

Description

I've created a Model following this Example:
GrammarJsonResponse.cs

I'm using the provided grammar, which forces the model to respond in JSON

This works great for the first prompt, but for any subsequent prompt, the response is just Empty

Reproduction Steps

• Execute the Code in Example:
  GrammarJsonResponse.cs

• Write 2 Prompts

=> See: second Prompt is Empty

Environment & Configuration

• Operating system: Windows 11
• .NET runtime version: .NET 8
• LLamaSharp version: 0.15.0
• CUDA version (if you are using cuda backend): 12
• CPU & GPU device: 13th Gen Intel(R) Core(TM) i7-13850HX, 2100 Mhz, 20 Core(s), 28 Logical Processor(s) & Nvidia RTX 4000 Ada Generation Laptop GPU

Known Workarounds

No response

[martindevans]

Ah that looks like a bug in the example.

A "grammar instance" (created on line 28) is what contains the state in a grammar, so after one response that instance has reached the end and there are no valid tokens to be sampled!

Every time you want to start a new response you need a fresh grammar instance, to reset that state back to the start.

If you'd like to submit a PR to fix that I'll be happy to review and merge it for you :)

[L-Wehmschulte]

Hey @martindevans
Thanks a lot. I'll try that out and definitely file a pr to fix that sample latest on Monday

[martindevans]

Resolved by #908. Thanks for the PR ❤️