Why is my Deepseek 2 responding multiple times #1103
Unanswered
vijaysridhara
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
I am trying to use the RichardErkhov/deepseek-ai_-_DeepSeek-V2-Lite-Chat-gguf, however the output is redundant. Can you please suggest any right parameters, though I played with multiple options?
`
//Inference parameters (Kept MaxTokens lower than ContextSize)
InferenceParams inferenceParams = new InferenceParams()
{
MaxTokens = 128,
SamplingPipeline = new DefaultSamplingPipeline
{
Temperature = 0.3f,
},
AntiPrompts = new List() { "User:" }
};
//Some other parameters
this.parameters = new ModelParams(modelPath)
{
ContextSize = 256,
GpuLayerCount = 2
};
`

Beta Was this translation helpful? Give feedback.
All reactions