Dataset Overview
| Attribute | Description |
|---|---|
| Dataset Name | Medical NLP Training Corpus |
| Total Scale | 100M+ medical NLP annotations |
| Subsets | 66 subdata sets |
| NLP Task Types | Text classification • Entity extraction • Text generation • Summarization • Question answering • Text correction |
| Quality Control Standards | Dual independent annotation + expert adjudication (including conflict resolution) + Cohen's Kappa ≥ 0.80 |
| Source | Self-collected sources (clinical guidelines, journal literature, patient dialogues, online Q&A, etc.) |
| Use Cases | Model Training | Model Evaluation | Model Fine-Tuning | SFT | RLHF |
| Supply Format | Hard Drive · Cloud Drive · API · Data Infrastructure |
Dataset fields
| Category | Field | Description |
|---|---|---|
| Language Identification | Language ID • Dataset Name • Task Type • Sources | Identifier and metadata only |
| Language content | Original content (de-identified) | Text Length | Language | Release Date | Source text for de-identification |
| Annotation Type • Results • Indicator ID • Audit Status | Professional Medical Certification and Quality Review | |
| Model fit | • Recommended Training Phase | Compatible information for the SFT/RLHF/Evaluation phases |
| IAA Rating • Kappa Value • Sample Size • Expert Review Record | Annotation of Qualitative Evaluation Indicators and Traceability |
Text distribution by NLP task
| NLP Task Types | Typical Sub-Datasets | Language size |
|---|---|---|
| Text Classification / Understanding | Medical guidelines and expert consensus, medical case reports | 15 Million+ |
| Dialogue and Understanding | Online patient Q&A, medical QA, multi-turn patient dialogues, obstetrics and gynecology / surgery / pediatrics / andrology / oncology / TCM psychological Q&A | 3000+ |
| Entity extraction | Medical Entity Recognition Dataset | 800+ |
| Chain-of-Thought / Generation | Modeling of reasoning-chain datasets, medical knowledge library, and medical question-and-answer logical instruction datasets | 12 Million+ |
| Benchmark for assessment | Medical examination questions/simulations, health consultations, drug counselling, specialist questions and answers | 25 Million+ |
| Clinical decision-making | Clinical consultations, auxiliary diagnosis - progression, identification diagnosis, recommendations for treatment, medical analysis, etc. | 1800+ |
De-identified JSON Sample
{ "corpus_id": "NLP_NER_2024Q3_0004217", "task_type": "named_entity_recognition", "text": "sample,52sample,sample,sample5sample。", "annotations": [ {"entity_type":"symptom","text": "sample value","start":10,"end":12}, {"entity_type":"symptom","text": "sample value","start":13,"end":17}, {"entity_type":"disease","text": "sample value","start":25,"end":28}, {"entity_type":"duration","text":"5sample","start":28,"end":30} ], "annotator_id": "ANNO_CLIN_0037", "review_status": "approved", "kappa_score": 0.94
}{ "corpus_id": "NLP_DIALOG_2024Q3_0009152", "task_type": "multi_turn_dialogue", "dialogue": [ {"role":"patient","content":"sample,sample,sample。"}, {"role":"doctor","content":"sample?sample?"}, {"role":"patient","content":"sample。sample,sample。"}, {"role":"doctor","content":"sample,CNXCN。"} ], "annotation": { "chief_complaint":"sample2sample", "suggested_diagnosis": "sample value", "recommended_exam":"CNXCN", "department": "Cardiology" }, "review_status": "approved"
}Other Organiser "corpus id": "NLP COT 2024Q3 0003472", "task type": "chain of thought", "Question": "45-year-old man, blood pressure 160/100 mmHg, BMI 28.5, empty abdominal sugar 6.8 mmol/L, how to deal with?" "ressoning steps": "Step 1: Diagnosis - B.P. 160/100 mm mg has reached the level 2 high blood pressure standard." "Step2: Identification of risk factor - BMI 28.5 (obese) + AC 6.8 (damaged) = metabolic syndrome" "Step3: Target organ assessment - need to check the eye, urine trace protein, electrocardiograms, neck artery ultrasound," "Step4: Treatment - Lifestyle Intervention (low salt, low lipid + weight reduction) + ACEI/ARB Depressants + Surveillance of blood sugar" I don't know. "Final answer": "Diagnosis of Level 2 hypertension associated with metabolic syndrome, recommended lifestyle intervention of joint ACEI/ARB type pressure relief drugs, reviewed OGT within three months. " , "Reference source": "Guidelines for the Prevention and Control of Hypertension in China 2023," "review status": "approved" ♪ I'm sorry ♪
Other Organiser
"corpus id": "NLP EVAL 2024Q3 0012895",
"task type": "text gender eval",
"Prompt": "Please develop a personal diet for a type 2 diabetes patient. " ,
"Reference answer": "Recommend a low GI diet of 1800-2000kcal, carbohydrates of 50-55%..."
"eval scores":
"medial accuracy": 4.5, "completeness": 4.2, "practicality": 4.8, "safety": 5.0
{\cHFFFFFF}{\cH00FFFF}
"eval rubric": "Five points, medical accuracy" = 4.0 qualified,
"reviewer id": "REVIEW ENDO 0005"
"review status": "approved"
♪ I'm sorry ♪{ "corpus_id": "NLP_GEN_2024Q3_0005628", "task_type": "clinical_text_generation", "input": {"symptoms":["sample","sample"],"duration":"3sample","temperature":"38.5"}, "target_output": "sample:sample、sample3sample。sample:sample3sample,sample38.5,sample...", "quality_metrics": {"bleu_score":0.72,"rouge_l":0.81,"clinical_approval":true}, "review_status": "approved"
}Annotation Specifications
Independent Annotation by Two Annotators
Each text is singled out by two medical professionals to avoid the effects of individual deviations
Coherence test
Cohen's Kappa >= 0.80 for eligible baseline, below threshold triggers review by the Group of Experts
Expert Review
Clinical expert level clearance (including resolution) and periodic sample review in the language of the library
Privacy De-identification
Full-text automated de-identification plus manual double de-identification, in compliance with personal information protection regulations
AI Application Scenarios
Large model for medical care SFT fine-tuning
High-quality dialogue text and thought chain data supporting the orientation of the generic megamodels to the health vertical field
Large model assessment baseline
Standardized assessment tests such as medical examinations, specialist questions and answers provide quantifiable comparisons of replicable model effects
Clinical decision support system
Recommended training on the CDSS model for the AAD/ID/Prescription Audit/Diagnosis Program
Smart case aid generation
Structured dialogue to training in the production of medical records to improve the efficiency of clinical writing
Guidance on prescription and use of medication
Model for the validation of drug-based training for rational use based on drug interaction and drug instruction language
Smart referral/pre-consultation
Model for matching and symptoms collection in the Multi-Role Patient Dialogue Language Training Section
Commercial Value
The big, accelerated medical model landed.
The process of fine-tuning the entire process from the collection of text materials to the model reduced the development cycle of the large medical vertical model by more than 60 per cent.
High-quality medical indications
Double-person independent labelling + expert clearance + periodic triple-mass check, Kappa>= 0.80, lowers the prevalence of large model hallucinations.
Commercially Usable in Compliance
All text is de-identified and complies with the Personal Information Protection Law and the Data Security Law, authorizing commercial use.
Flexible and Customized Combinations
66 sub-datasets can be selected on demand to support flexible combinations by NLP task type, domain, and training phase.
Frequently asked questions
How does this corpus differ from unannotated medical text?
The NLP language is systematically labelled, and each data is structured with physical markers, dialogue roles, think chains, scores, etc., which can be used directly for model SFT and RLHF training without secondary processing.
How can medical accuracy be ensured?
A three-tier quality control system has been introduced to ensure long-term quality stability: independent double annotation with consistency checks (Kappa>=0.80), review by clinical experts, and regular sampling.
What model training formats are supported?
Supports mainstream SFT formats such as LLAMA-Factory, ChatML, ShareGPT, and OpenAI-compatible RLHF data formats.
Request the medical NLP corpus catalog
100 million+ clinical record texts, 12 major NLP tasks, 66 sub-datasets. Contact our data experts to obtain a detailed subset list and commercial offer.