Get in touch
Work location
B-810/811, World Trade Tower (WTT), Makarba, Sarkhej Gandhinagar Highway, Ahmedabad,
Gujarat, India, 380015
Social link
A general-purpose model sounds confident answering a specialized question even when it’s wrong in ways a domain expert would catch immediately. Closing that gap is usually cheaper than it sounds, and doesn’t require retraining a model from scratch.
DomainTune is a LoRA adapter trained for medical-domain terminology on a GPT-3.5-turbo base. It fine-tunes a small set of adapter parameters on top of a frozen base model instead of retraining the whole thing, and ships with a playground that puts the base model’s answer next to the adapted one, so the difference is visible, not just claimed.
A general-purpose model answers a specialized question with the right tone of confidence and the wrong depth of knowledge, and full fine-tuning to fix that is too expensive to justify for most single-domain use cases.
LoRA and QLoRA train a small adapter on top of a frozen base model, adapting it to domain vocabulary at a fraction of the compute a full retrain needs, backed by a side-by-side evaluation so the improvement is demonstrated, not assumed.
Full fine-tuning updates every parameter in the model, which rarely clears the bar for one domain’s vocabulary problem. LoRA and QLoRA update a small set of adapter weights layered on a base model that stays frozen; here, a GPT-3.5-turbo checkpoint adapted for medical terminology. The adapter is swappable, so the same base can carry another domain’s adapter without starting over. Ask both versions about LoRA’s benefits in the medical domain: the base gives a generic paragraph, while the fine-tuned checkpoint names five specific, correctly ordered benefits.
Four stages take a domain from raw examples to a model a team can trust: curate the data, train the adapter on a frozen base, evaluate it against the untouched model, and let stakeholders compare the two for themselves.
Domain-specific examples are collected and cleaned into the fine-tuning set. This step took longer than the training itself.
.svg)
Adapter weights train on top of the frozen base model, touching a small fraction of total parameters.
Held-out examples compare the adapted model’s answers directly against the untouched base model’s.
An interactive chat where a team can run the same prompt against both versions and see the difference themselves.
Only the LoRA weights update. The base model’s own parameters never change, which is also why the adapter is swappable.
No full retrain means the training itself doesn’t need a GPU cluster to be worth doing.
The same frozen base model can carry a different adapter for a different domain without starting from scratch.
Base and fine-tuned responses sit next to each other on the same held-out prompts. The comparison is the deliverable.
Adapters ship in a standard Hugging Face format rather than something custom only this pipeline can load.
.svg)
The frozen base is never overwritten, so a team can always check the adapted model against the original.
The aim was a model that knows the domain, with proof a stakeholder can see for themselves. Here is what changed on the same prompts.
Medical terminology specifically, not healthcare broadly, because a narrower target produces a cleaner training set.
The first pass was noisy enough that an early adapter did worse than the base on some held-out prompts. Curation ran longer than expected.
Once the data was clean, training took hours, not the days a full fine-tune would have needed.
Fixed the evaluation set before training finished, so there was an honest bar to clear rather than cherry-picked examples.
Made so a non-technical stakeholder could see the difference themselves instead of taking a score on faith.
Deliberately ran unrelated prompts so the limits of the narrow gain were known upfront, not discovered by a confused user.
