Classic RAG will let you down here. You need a system that maps entities via relationships like a Knowledge Graph. The paper you read yesterday about RAPTOR, HYDE or some new "hot

Local LLMs are wonderful, and we all know that, but something that's always bothered me is that nobody in the scene seems to want to standardize or even investigate the flaws of th

Oct 19, 2023 · Hey yall For some time I have been interested in applications using a single base model and multiple LoRAs during inference. Specifically I would like to run a setu

Jun 27, 2023 · Thanks to our most esteemed model trainer, Mr TheBloke, we now have versions of Manticore, Nous Hermes (!!), WizardLM and so on, all with SuperHOT 8k context LoRA.

Jun 9, 2023 · Man, Falcon’s been on top for a hot minute now. Anybody know if there’s a ggml based 4bit version? I’m super happy with the uncensored CoT Storytelling model,

But the reality is that the current Phi model series is hot garbage with extremely limited practical use cases, relative to other options...unless you're just trying to hack benchm

Posted by u/DontPlanToEnd - 96 votes and 51 comments

Yi runs HOT. Personally I run 0.8 with 0.05 MinP and all other samplers disabled, but Mirostat with low Tau also works. Also, set repetition penalty to 1.05-1.2ish. I am open to sa

193 votes, 98 comments. true ....appropriate targets depending on your requirements. Remember that running build commands can modify your project files and potentially create new f

Feb 13, 2024 · It’s really not that hot. Running Code Wizard 70b doesn’t break 600watts and I’m trying to push it … each GPU idles around 8 W and when running the model, t