Mechanics Diagrams
Force diagrams, velocity-time graphs, phase diagrams, oscillation waveforms, wave diagrams.
Evidence: MMMU Physics 56.0% — diagram-heavy categories far below average.
Subject Overview
25%
of Total Question Bank
2nd largest discipline
4
Sub-Domains
Physics · Chemistry · Biology · Earth Sciences
6
Diagram Types
Circuit · Force · Wave · Field · Ray · Micrograph
Science was among the first domains where multimodal LLMs made breakthroughs — MMMU reports GPT-4o at 56.0% on Physics and 50.0% on Chemistry. However, the SeePhys benchmark (1,000 diagram-based physics questions) reveals that Qwen2.5-VL 7B achieves only 40.7% across 7 diagram types, exposing the true difficulty of diagrammatic reasoning. Langhui Science benchmark emphasizes high-diagram-density question types: experiment setups (circuits, apparatus), physics diagrams (force, wave, field, ray), chemical structures and reaction equations, micrographs (cell, tissue, section), maps, and remote sensing imagery.
Sub-Domains & High-Value Difficulty Zones
Physics (40%)
Electromagnetism
Circuit diagrams, electromagnetic field lines, Ampere force direction, Lenz's law applications.
Optics & Modern Physics
Ray diagrams, interference/diffraction patterns, atomic energy level diagrams, semiconductor band diagrams.
Evidence: SeePhys 7 diagram types — Qwen2.5-VL 7B at 40.7%.
Thermodynamic Cycle Diagrams
P-V diagrams, T-S diagrams, cycle efficiency calculations, Carnot cycle interpretation.
Chemistry (30%)
Molecular Structure & Bonding
VSEPR geometry prediction, molecular orbital diagrams, hybridization visualization.
Reaction Mechanisms
Arrow-pushing diagrams, energy profile diagrams, catalytic cycle schematics.
Spectroscopy & Analytical Chemistry
IR/NMR/MS spectrum interpretation, chromatograms, titration curves.
Evidence: MMMU Chemistry 50.0%.
Biology (20%)
Cell & Molecular Biology
Electron micrographs, organelle identification, signaling pathway diagrams.
Genetics & Evolution
Pedigree charts, phylogenetic trees, Punnett squares, gel electrophoresis.
Ecology & Environmental Science
Food web diagrams, population growth curves, biome distribution maps.
Earth Sciences (10%)
Geology & Geophysics
Stratigraphic columns, seismic wave diagrams, plate tectonics maps, mineral identification images.
Meteorology & Oceanography
Weather maps, satellite imagery, ocean current diagrams, climate model visualizations.
Difficulty Distribution
| Level | Stage | Share | Typical Question Types |
|---|---|---|---|
| L1 Lower Undergrad | Freshman-Sophomore | 15% | Basic formula application, single-concept diagrams |
| L2 Upper Undergrad | Junior-Senior | 45% | Multi-step derivations, complex diagrams, experiment analysis |
| L3 Graduate Entry | GRE / Qualifying exams | 30% | Cross-domain synthesis, research-level reasoning |
| L4 Research | PhD / Frontier research | 10% | Paper-level proofs, advanced modeling |
Sample Questions
STEM-2026-SCI-0000012L1 Lower Undergrad
Mechanics · Force Diagram Analysis
Prompt: A block of mass m rests on an inclined plane at angle θ with friction coefficient μ. The accompanying diagram shows the force decomposition. Determine the condition for static equilibrium and the direction of the friction force.
Answer: Static equilibrium requires μ ≥ tanθ. Friction acts upward along the incline (opposing the component of gravity mg·sinθ).
5-Model Evaluation
GPT-5.1
2/5
Claude Opus 4.6
2/5
Gemini-3.1-Pro
3/5
Qwen3.6-Plus
1/5
DeepSeek-V4
1/5
STEM-2026-SCI-0000013L2 Upper Undergrad
Chemistry · IR Spectrum Interpretation
Prompt: An IR spectrum shows a strong broad peak at ~3300 cm⁻¹, a sharp peak at ~1700 cm⁻¹, and peaks at ~1600 and ~1500 cm⁻¹. Identify the functional groups present and propose the most likely compound class.
Answer: 3300 cm⁻¹ (broad) = O-H stretch (carboxylic acid); 1700 cm⁻¹ = C=O stretch; 1600+1500 cm⁻¹ = aromatic C=C. Most likely: an aromatic carboxylic acid.
5-Model Evaluation
GPT-5.1
2/5
Claude Opus 4.6
1/5
Gemini-3.1-Pro
1/5
Qwen3.6-Plus
0/5
DeepSeek-V4
1/5
Delivery & API
JSONL Bulk Download: Full fields + image URLs + 5-model evaluation + reasoning chains.
REST API: GET /DataAssetsAPI/stem-questions/science. For enterprise API keys, contact lk@langhuiai.com.
FAQ
What is the difficulty threshold for the Science benchmark?
Each question must achieve ≤40% pass rate across 5 models × 5 passes (GPT-5.1 / Claude Opus 4.6 / Gemini-3.1-Pro / Qwen3.6-Plus / DeepSeek-V4). Human expert accuracy ≥95%, with dual-review gold standard.
What sub-domains does the Science benchmark cover?
Physics (mechanics/electromagnetism/optics/modern physics/thermodynamics), Chemistry, Biology (cell biology/genetics/ecology/microscopy), and Earth Sciences.
What diagram types does the Science benchmark emphasize?
Experiment diagrams (circuits, apparatus), physics diagrams (force/wave/field/ray), chemical structures and equations, micrographs, maps, and remote sensing imagery.
Is there overlap with public benchmarks?
Zero overlap. <0.5% pHash+n-gram fingerprint overlap with MMMU, SeePhys, ScienceQA, and 5 other public benchmarks.
How is answer quality validated?
All questions undergo dual-review by subject-matter experts. Physics/chemistry numerical answers include step-by-step derivations verified against multiple textbook sources.