feat(tests): CI-enforce skill authoring standards; clear all remaining debt
New tests/skills/test_authoring_standards.py parametrizes every bundled + optional SKILL.md (1148 checks) against the mechanically-verifiable subset of the hardline standards: - required frontmatter fields (name/description/version/author/license/ platforms) + tags - frontmatter name == directory name - description <= 60 chars, ends with period, no marketing words - related_skills resolve in-repo - no machine-local paths - <= 100k chars Grandfather dict for legacy debt ships EMPTY — all pre-existing violations fixed in this PR: - 13 frontmatter names canonicalized to their directory names (the install identifier); all related_skills references updated (comfyui -> stable- diffusion). Fixes the class behind PR #42788's report; also fixes here.now's invalid dot-name. - optional-skills/devops/cli -> inference-sh-cli (dir was the generic 'cli'; fm name was right) incl. docs pages (en + zh-Hans), catalog row, sidebar entry. - pytorch-fsdp: 157k generated 'Quick Reference' dump moved to references/common-patterns.md; SKILL.md 159k -> 2.5k with a pointer. - research-paper-writing: 31.7k Phase 5 drafting section moved to references/phase5-paper-drafting.md; SKILL.md 103k -> 71k. Docs regenerated with scope discipline.
This commit is contained in:
teknium1
Teknium
parent
3898e646e5
commit
55982159dd
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: huggingface-accelerate
|
||||
name: accelerate
|
||||
description: Run PyTorch training across GPUs with minimal changes.
|
||||
version: 1.0.1
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: optimizing-attention-flash
|
||||
name: flash-attention
|
||||
description: Speed up long-sequence transformer training and inference.
|
||||
version: 1.0.1
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: lambda-labs-gpu-cloud
|
||||
name: lambda-labs
|
||||
description: On-demand GPU cloud instances for ML training.
|
||||
version: 1.0.0
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: modal-serverless-gpu
|
||||
name: modal
|
||||
description: Serverless GPU cloud for ML jobs and model APIs.
|
||||
version: 1.0.1
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: peft-fine-tuning
|
||||
name: peft
|
||||
description: Fine-tune large LLMs with LoRA on limited GPU memory.
|
||||
version: 1.0.0
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: qdrant-vector-search
|
||||
name: qdrant
|
||||
description: Vector search engine for production RAG systems.
|
||||
version: 1.0.1
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: sparse-autoencoder-training
|
||||
name: saelens
|
||||
description: Train sparse autoencoders to interpret model features.
|
||||
version: 1.0.1
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: simpo-training
|
||||
name: simpo
|
||||
description: Reference-free preference alignment, simpler than DPO.
|
||||
version: 1.0.0
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: slime-rl-training
|
||||
name: slime
|
||||
description: RL post-training for LLMs with Megatron and SGLang.
|
||||
version: 1.0.0
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: stable-diffusion-image-generation
|
||||
name: stable-diffusion
|
||||
description: Text-to-image generation, inpainting, and img2img.
|
||||
version: 1.0.0
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: distributed-llm-pretraining-torchtitan
|
||||
name: torchtitan
|
||||
description: Pretrain LLMs at scale with PyTorch 4D parallelism.
|
||||
version: 1.0.1
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: fine-tuning-with-trl
|
||||
name: trl-fine-tuning
|
||||
description: "TRL: SFT, DPO, GRPO, RLOO reward modeling for LLM RLHF."
|
||||
version: 1.0.1
|
||||
author: Orchestra Research
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: here.now
|
||||
name: here-now
|
||||
description: Publish sites to {slug}.here.now and store files in Drives.
|
||||
version: 1.15.3
|
||||
author: here.now
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ metadata:
|
|||
- creative
|
||||
- generative-ai
|
||||
- video-generation
|
||||
related_skills: [stable-diffusion-image-generation]
|
||||
related_skills: [stable-diffusion]
|
||||
category: creative
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -777,760 +777,10 @@ See [references/autoreason-methodology.md](references/autoreason-methodology.md)
|
|||
|
||||
## Phase 5: Paper Drafting
|
||||
|
||||
**Goal**: Write a complete, publication-ready paper.
|
||||
|
||||
### Context Management for Large Projects
|
||||
|
||||
A paper project with 50+ experiment files, multiple result directories, and extensive literature notes can easily exceed the agent's context window. Manage this proactively:
|
||||
|
||||
**What to load into context per drafting task:**
|
||||
|
||||
| Drafting Task | Load Into Context | Do NOT Load |
|
||||
|---------------|------------------|-------------|
|
||||
| Writing Introduction | `experiment_log.md`, contribution statement, 5-10 most relevant paper abstracts | Raw result JSONs, full experiment scripts, all literature notes |
|
||||
| Writing Methods | Experiment configs, pseudocode, architecture description | Raw logs, results from other experiments |
|
||||
| Writing Results | `experiment_log.md`, result summary tables, figure list | Full analysis scripts, intermediate data |
|
||||
| Writing Related Work | Organized citation notes (Step 1.4 output), .bib file | Experiment files, raw PDFs |
|
||||
| Revision pass | Full paper draft, specific reviewer concerns | Everything else |
|
||||
|
||||
**Principles:**
|
||||
- **`experiment_log.md` is the primary context bridge** — it summarizes everything needed for writing without loading raw data files (see Step 4.6)
|
||||
- **Load one section's context at a time** when delegating. A sub-agent drafting Methods doesn't need the literature review notes.
|
||||
- **Summarize, don't include raw files.** For a 200-line result JSON, load a 10-line summary table. For a 50-page related paper, load the 5-sentence abstract + your 2-line note about its relevance.
|
||||
- **For very large projects**: Create a `context/` directory with pre-compressed summaries:
|
||||
```
|
||||
context/
|
||||
contribution.md # 1 sentence
|
||||
experiment_summary.md # Key results table (from experiment_log.md)
|
||||
literature_map.md # Organized citation notes
|
||||
figure_inventory.md # List of figures with descriptions
|
||||
```
|
||||
|
||||
### The Narrative Principle
|
||||
|
||||
**The single most critical insight**: Your paper is not a collection of experiments — it's a story with one clear contribution supported by evidence.
|
||||
|
||||
Every successful ML paper centers on what Neel Nanda calls "the narrative": a short, rigorous, evidence-based technical story with a takeaway readers care about.
|
||||
|
||||
**Three Pillars (must be crystal clear by end of introduction):**
|
||||
|
||||
| Pillar | Description | Test |
|
||||
|--------|-------------|------|
|
||||
| **The What** | 1-3 specific novel claims | Can you state them in one sentence? |
|
||||
| **The Why** | Rigorous empirical evidence | Do experiments distinguish your hypothesis from alternatives? |
|
||||
| **The So What** | Why readers should care | Does this connect to a recognized community problem? |
|
||||
|
||||
**If you cannot state your contribution in one sentence, you don't yet have a paper.**
|
||||
|
||||
### The Sources Behind This Guidance
|
||||
|
||||
This skill synthesizes writing philosophy from researchers who have published extensively at top venues. The writing philosophy layer was originally compiled by [Orchestra Research](https://github.com/orchestra-research) as the `ml-paper-writing` skill.
|
||||
|
||||
| Source | Key Contribution | Link |
|
||||
|--------|-----------------|------|
|
||||
| **Neel Nanda** (Google DeepMind) | The Narrative Principle, What/Why/So What framework | [How to Write ML Papers](https://www.alignmentforum.org/posts/eJGptPbbFPZGLpjsp/highly-opinionated-advice-on-how-to-write-ml-papers) |
|
||||
| **Sebastian Farquhar** (DeepMind) | 5-sentence abstract formula | [How to Write ML Papers](https://sebastianfarquhar.com/on-research/2024/11/04/how_to_write_ml_papers/) |
|
||||
| **Gopen & Swan** | 7 principles of reader expectations | [Science of Scientific Writing](https://cseweb.ucsd.edu/~swanson/papers/science-of-writing.pdf) |
|
||||
| **Zachary Lipton** | Word choice, eliminating hedging | [Heuristics for Scientific Writing](https://www.approximatelycorrect.com/2018/01/29/heuristics-technical-scientific-writing-machine-learning-perspective/) |
|
||||
| **Jacob Steinhardt** (UC Berkeley) | Precision, consistent terminology | [Writing Tips](https://bounded-regret.ghost.io/) |
|
||||
| **Ethan Perez** (Anthropic) | Micro-level clarity tips | [Easy Paper Writing Tips](https://ethanperez.net/easy-paper-writing-tips/) |
|
||||
| **Andrej Karpathy** | Single contribution focus | Various lectures |
|
||||
|
||||
**For deeper dives into any of these, see:**
|
||||
- [references/writing-guide.md](references/writing-guide.md) — Full explanations with examples
|
||||
- [references/sources.md](references/sources.md) — Complete bibliography
|
||||
|
||||
### Time Allocation
|
||||
|
||||
Spend approximately **equal time** on each of:
|
||||
1. The abstract
|
||||
2. The introduction
|
||||
3. The figures
|
||||
4. Everything else combined
|
||||
|
||||
**Why?** Most reviewers form judgments before reaching your methods. Readers encounter your paper as: title → abstract → introduction → figures → maybe the rest.
|
||||
|
||||
### Writing Workflow
|
||||
|
||||
```
|
||||
Paper Writing Checklist:
|
||||
- [ ] Step 1: Define the one-sentence contribution
|
||||
- [ ] Step 2: Draft Figure 1 (core idea or most compelling result)
|
||||
- [ ] Step 3: Draft abstract (5-sentence formula)
|
||||
- [ ] Step 4: Draft introduction (1-1.5 pages max)
|
||||
- [ ] Step 5: Draft methods
|
||||
- [ ] Step 6: Draft experiments & results
|
||||
- [ ] Step 7: Draft related work
|
||||
- [ ] Step 8: Draft conclusion & discussion
|
||||
- [ ] Step 9: Draft limitations (REQUIRED by all venues)
|
||||
- [ ] Step 10: Plan appendix (proofs, extra experiments, details)
|
||||
- [ ] Step 11: Complete paper checklist
|
||||
- [ ] Step 12: Final review
|
||||
```
|
||||
|
||||
### Two-Pass Refinement Pattern
|
||||
|
||||
When drafting with an AI agent, use a **two-pass** approach (proven effective in SakanaAI's AI-Scientist pipeline):
|
||||
|
||||
**Pass 1 — Write + immediate refine per section:**
|
||||
For each section, write a complete draft, then immediately refine it in the same context. This catches local issues (clarity, flow, completeness) while the section is fresh.
|
||||
|
||||
**Pass 2 — Global refinement with full-paper context:**
|
||||
After all sections are drafted, revisit each section with awareness of the complete paper. This catches cross-section issues: redundancy, inconsistent terminology, narrative flow, and gaps where one section promises something another doesn't deliver.
|
||||
|
||||
```
|
||||
Second-pass refinement prompt (per section):
|
||||
"Review the [SECTION] in the context of the complete paper.
|
||||
- Does it fit with the rest of the paper? Are there redundancies with other sections?
|
||||
- Is terminology consistent with Introduction and Methods?
|
||||
- Can anything be cut without weakening the message?
|
||||
- Does the narrative flow from the previous section and into the next?
|
||||
Make minimal, targeted edits. Do not rewrite from scratch."
|
||||
```
|
||||
|
||||
### LaTeX Error Checklist
|
||||
|
||||
Append this checklist to every refinement prompt. These are the most common errors when LLMs write LaTeX:
|
||||
|
||||
```
|
||||
LaTeX Quality Checklist (verify after every edit):
|
||||
- [ ] No unenclosed math symbols ($ signs balanced)
|
||||
- [ ] Only reference figures/tables that exist (\ref matches \label)
|
||||
- [ ] No fabricated citations (\cite matches entries in .bib)
|
||||
- [ ] Every \begin{env} has matching \end{env} (especially figure, table, algorithm)
|
||||
- [ ] No HTML contamination (</end{figure}> instead of \end{figure})
|
||||
- [ ] No unescaped underscores outside math mode (use \_ in text)
|
||||
- [ ] No duplicate \label definitions
|
||||
- [ ] No duplicate section headers
|
||||
- [ ] Numbers in text match actual experimental results
|
||||
- [ ] All figures have captions and labels
|
||||
- [ ] No overly long lines that cause overfull hbox warnings
|
||||
```
|
||||
|
||||
### Step 5.0: Title
|
||||
|
||||
The title is the single most-read element of the paper. It determines whether anyone clicks through to the abstract.
|
||||
|
||||
**Good titles**:
|
||||
- State the contribution or finding: "Autoreason: When Iterative LLM Refinement Works and Why It Fails"
|
||||
- Highlight a surprising result: "Scaling Data-Constrained Language Models" (implies you can)
|
||||
- Name the method + what it does: "DPO: Direct Preference Optimization of Language Models"
|
||||
|
||||
**Bad titles**:
|
||||
- Too generic: "An Approach to Improving Language Model Outputs"
|
||||
- Too long: anything over ~15 words
|
||||
- Jargon-only: "Asymptotic Convergence of Iterative Stochastic Policy Refinement" (who is this for?)
|
||||
|
||||
**Rules**:
|
||||
- Include your method name if you have one (for citability)
|
||||
- Include 1-2 keywords reviewers will search for
|
||||
- Avoid colons unless both halves carry meaning
|
||||
- Test: would a reviewer know the domain and contribution from the title alone?
|
||||
|
||||
### Step 5.1: Abstract (5-Sentence Formula)
|
||||
|
||||
From Sebastian Farquhar (DeepMind):
|
||||
|
||||
```
|
||||
1. What you achieved: "We introduce...", "We prove...", "We demonstrate..."
|
||||
2. Why this is hard and important
|
||||
3. How you do it (with specialist keywords for discoverability)
|
||||
4. What evidence you have
|
||||
5. Your most remarkable number/result
|
||||
```
|
||||
|
||||
**Delete** generic openings like "Large language models have achieved remarkable success..."
|
||||
|
||||
### Step 5.2: Figure 1
|
||||
|
||||
Figure 1 is the second thing most readers look at (after abstract). Draft it before writing the introduction — it forces you to clarify the core idea.
|
||||
|
||||
| Figure 1 Type | When to Use | Example |
|
||||
|---------------|-------------|---------|
|
||||
| **Method diagram** | New architecture or pipeline | TikZ flowchart showing your system |
|
||||
| **Results teaser** | One compelling result tells the whole story | Bar chart: "Ours vs baselines" with clear gap |
|
||||
| **Problem illustration** | The problem is unintuitive | Before/after showing failure mode you fix |
|
||||
| **Conceptual diagram** | Abstract contribution needs visual grounding | 2x2 matrix of method properties |
|
||||
|
||||
**Rules**: Figure 1 must be understandable without reading any text. The caption alone should communicate the core idea. Use color purposefully — don't just decorate.
|
||||
|
||||
### Step 5.3: Introduction (1-1.5 pages max)
|
||||
|
||||
Must include:
|
||||
- Clear problem statement
|
||||
- Brief approach overview
|
||||
- 2-4 bullet contribution list (max 1-2 lines each in two-column format)
|
||||
- Methods should start by page 2-3
|
||||
|
||||
### Step 5.4: Methods
|
||||
|
||||
Enable reimplementation:
|
||||
- Conceptual outline or pseudocode
|
||||
- All hyperparameters listed
|
||||
- Architectural details sufficient for reproduction
|
||||
- Present final design decisions; ablations go in experiments
|
||||
|
||||
### Step 5.5: Experiments & Results
|
||||
|
||||
For each experiment, explicitly state:
|
||||
- **What claim it supports**
|
||||
- How it connects to main contribution
|
||||
- What to observe: "the blue line shows X, which demonstrates Y"
|
||||
|
||||
Requirements:
|
||||
- Error bars with methodology (std dev vs std error)
|
||||
- Hyperparameter search ranges
|
||||
- Compute infrastructure (GPU type, total hours)
|
||||
- Seed-setting methods
|
||||
|
||||
### Step 5.6: Related Work
|
||||
|
||||
Organize methodologically, not paper-by-paper. Cite generously — reviewers likely authored relevant papers.
|
||||
|
||||
### Step 5.7: Limitations (REQUIRED)
|
||||
|
||||
All major conferences require this. Honesty helps:
|
||||
- Reviewers are instructed not to penalize honest limitation acknowledgment
|
||||
- Pre-empt criticisms by identifying weaknesses first
|
||||
- Explain why limitations don't undermine core claims
|
||||
|
||||
### Step 5.8: Conclusion & Discussion
|
||||
|
||||
**Conclusion** (required, 0.5-1 page):
|
||||
- Restate the contribution in one sentence (different wording from abstract)
|
||||
- Summarize key findings (2-3 sentences, not a list)
|
||||
- Implications: what does this mean for the field?
|
||||
- Future work: 2-3 concrete next steps (not vague "we leave X for future work")
|
||||
|
||||
**Discussion** (optional, sometimes combined with conclusion):
|
||||
- Broader implications beyond immediate results
|
||||
- Connections to other subfields
|
||||
- Honest assessment of when the method does and doesn't work
|
||||
- Practical deployment considerations
|
||||
|
||||
**Do NOT** introduce new results or claims in the conclusion.
|
||||
|
||||
### Step 5.9: Appendix Strategy
|
||||
|
||||
Appendices are unlimited at all major venues and are essential for reproducibility. Structure:
|
||||
|
||||
| Appendix Section | What Goes Here |
|
||||
|-----------------|---------------|
|
||||
| **Proofs & Derivations** | Full proofs too long for main text. Main text can state theorems with "proof in Appendix A." |
|
||||
| **Additional Experiments** | Ablations, scaling curves, per-dataset breakdowns, hyperparameter sensitivity |
|
||||
| **Implementation Details** | Full hyperparameter tables, training details, hardware specs, random seeds |
|
||||
| **Dataset Documentation** | Data collection process, annotation guidelines, licensing, preprocessing |
|
||||
| **Prompts & Templates** | Exact prompts used (for LLM-based methods), evaluation templates |
|
||||
| **Human Evaluation** | Annotation interface screenshots, instructions given to annotators, IRB details |
|
||||
| **Additional Figures** | Per-task breakdowns, trajectory visualizations, failure case examples |
|
||||
|
||||
**Rules**:
|
||||
- The main paper must be self-contained — reviewers are not required to read appendices
|
||||
- Never put critical evidence only in the appendix
|
||||
- Cross-reference: "Full results in Table 5 (Appendix B)" not just "see appendix"
|
||||
- Use `\appendix` command, then `\section{A: Proofs}` etc.
|
||||
|
||||
### Page Budget Management
|
||||
|
||||
When over the page limit:
|
||||
|
||||
| Cut Strategy | Saves | Risk |
|
||||
|-------------|-------|------|
|
||||
| Move proofs to appendix | 0.5-2 pages | Low — standard practice |
|
||||
| Condense related work | 0.5-1 page | Medium — may miss key citations |
|
||||
| Combine tables with subfigures | 0.25-0.5 page | Low — often improves readability |
|
||||
| Use `\vspace{-Xpt}` sparingly | 0.1-0.3 page | Low if subtle, high if obvious |
|
||||
| Remove qualitative examples | 0.5-1 page | Medium — reviewers like examples |
|
||||
| Reduce figure sizes | 0.25-0.5 page | High — figures must remain readable |
|
||||
|
||||
**Do NOT**: reduce font size, change margins, remove required sections (limitations, broader impact), or use `\small`/`\footnotesize` for main text.
|
||||
|
||||
### Step 5.10: Ethics & Broader Impact Statement
|
||||
|
||||
Most venues now require or strongly encourage an ethics/broader impact statement. This is not boilerplate — reviewers read it and can flag ethics concerns that trigger desk rejection.
|
||||
|
||||
**What to include:**
|
||||
|
||||
| Component | Content | Required By |
|
||||
|-----------|---------|-------------|
|
||||
| **Positive societal impact** | How your work benefits society | NeurIPS, ICML |
|
||||
| **Potential negative impact** | Misuse risks, dual-use concerns, failure modes | NeurIPS, ICML |
|
||||
| **Fairness & bias** | Does your method/data have known biases? | All venues (implicitly) |
|
||||
| **Environmental impact** | Compute carbon footprint for large-scale training | ICML, increasingly NeurIPS |
|
||||
| **Privacy** | Does your work use or enable processing of personal data? | ACL, NeurIPS |
|
||||
| **LLM disclosure** | Was AI used in writing or experiments? | ICLR (mandatory), ACL |
|
||||
|
||||
**Writing the statement:**
|
||||
|
||||
```latex
|
||||
\section*{Broader Impact Statement}
|
||||
% NeurIPS/ICML: after conclusion, does not count toward page limit
|
||||
|
||||
% 1. Positive applications (1-2 sentences)
|
||||
This work enables [specific application] which may benefit [specific group].
|
||||
|
||||
% 2. Risks and mitigations (1-3 sentences, be specific)
|
||||
[Method/model] could potentially be misused for [specific risk]. We mitigate
|
||||
this by [specific mitigation, e.g., releasing only model weights above size X,
|
||||
including safety filters, documenting failure modes].
|
||||
|
||||
% 3. Limitations of impact claims (1 sentence)
|
||||
Our evaluation is limited to [specific domain]; broader deployment would
|
||||
require [specific additional work].
|
||||
```
|
||||
|
||||
**Common mistakes:**
|
||||
- Writing "we foresee no negative impacts" (almost never true — reviewers distrust this)
|
||||
- Being vague: "this could be misused" without specifying how
|
||||
- Ignoring compute costs for large-scale work
|
||||
- Forgetting to disclose LLM use at venues that require it
|
||||
|
||||
**Compute carbon footprint** (for training-heavy papers):
|
||||
```python
|
||||
# Estimate using ML CO2 Impact tool methodology
|
||||
gpu_hours = 1000 # total GPU hours
|
||||
gpu_tdp_watts = 400 # e.g., A100 = 400W
|
||||
pue = 1.1 # Power Usage Effectiveness (data center overhead)
|
||||
carbon_intensity = 0.429 # kg CO2/kWh (US average; varies by region)
|
||||
|
||||
energy_kwh = (gpu_hours * gpu_tdp_watts * pue) / 1000
|
||||
carbon_kg = energy_kwh * carbon_intensity
|
||||
print(f"Energy: {energy_kwh:.0f} kWh, Carbon: {carbon_kg:.0f} kg CO2eq")
|
||||
```
|
||||
|
||||
### Step 5.11: Datasheets & Model Cards (If Applicable)
|
||||
|
||||
If your paper introduces a **new dataset** or **releases a model**, include structured documentation. Reviewers increasingly expect this, and NeurIPS Datasets & Benchmarks track requires it.
|
||||
|
||||
**Datasheets for Datasets** (Gebru et al., 2021) — include in appendix:
|
||||
|
||||
```
|
||||
Dataset Documentation (Appendix):
|
||||
- Motivation: Why was this dataset created? What task does it support?
|
||||
- Composition: What are the instances? How many? What data types?
|
||||
- Collection: How was data collected? What was the source?
|
||||
- Preprocessing: What cleaning/filtering was applied?
|
||||
- Distribution: How is the dataset distributed? Under what license?
|
||||
- Maintenance: Who maintains it? How to report issues?
|
||||
- Ethical considerations: Contains personal data? Consent obtained?
|
||||
Potential for harm? Known biases?
|
||||
```
|
||||
|
||||
**Model Cards** (Mitchell et al., 2019) — include in appendix for model releases:
|
||||
|
||||
```
|
||||
Model Card (Appendix):
|
||||
- Model details: Architecture, training data, training procedure
|
||||
- Intended use: Primary use cases, out-of-scope uses
|
||||
- Metrics: Evaluation metrics and results on benchmarks
|
||||
- Ethical considerations: Known biases, fairness evaluations
|
||||
- Limitations: Known failure modes, domains where model underperforms
|
||||
```
|
||||
|
||||
### Writing Style
|
||||
|
||||
**Sentence-level clarity (Gopen & Swan's 7 Principles):**
|
||||
|
||||
| Principle | Rule |
|
||||
|-----------|------|
|
||||
| Subject-verb proximity | Keep subject and verb close |
|
||||
| Stress position | Place emphasis at sentence ends |
|
||||
| Topic position | Put context first, new info after |
|
||||
| Old before new | Familiar info → unfamiliar info |
|
||||
| One unit, one function | Each paragraph makes one point |
|
||||
| Action in verb | Use verbs, not nominalizations |
|
||||
| Context before new | Set stage before presenting |
|
||||
|
||||
**Word choice (Lipton, Steinhardt):**
|
||||
- Be specific: "accuracy" not "performance"
|
||||
- Eliminate hedging: drop "may" unless genuinely uncertain
|
||||
- Consistent terminology throughout
|
||||
- Avoid incremental vocabulary: "develop", not "combine"
|
||||
|
||||
**Full writing guide with examples**: See [references/writing-guide.md](references/writing-guide.md)
|
||||
|
||||
### Using LaTeX Templates
|
||||
|
||||
**Always copy the entire template directory first, then write within it.**
|
||||
|
||||
```
|
||||
Template Setup Checklist:
|
||||
- [ ] Step 1: Copy entire template directory to new project
|
||||
- [ ] Step 2: Verify template compiles as-is (before any changes)
|
||||
- [ ] Step 3: Read the template's example content to understand structure
|
||||
- [ ] Step 4: Replace example content section by section
|
||||
- [ ] Step 5: Use template macros (check preamble for \newcommand definitions)
|
||||
- [ ] Step 6: Clean up template artifacts only at the end
|
||||
```
|
||||
|
||||
**Step 1: Copy the Full Template**
|
||||
|
||||
```bash
|
||||
cp -r templates/neurips2025/ ~/papers/my-paper/
|
||||
cd ~/papers/my-paper/
|
||||
ls -la # Should see: main.tex, neurips.sty, Makefile, etc.
|
||||
```
|
||||
|
||||
Copy the ENTIRE directory, not just the .tex file. Templates include style files (.sty), bibliography styles (.bst), example content, and Makefiles.
|
||||
|
||||
**Step 2: Verify Template Compiles First**
|
||||
|
||||
Before making ANY changes:
|
||||
```bash
|
||||
latexmk -pdf main.tex
|
||||
# Or manual: pdflatex main.tex && bibtex main && pdflatex main.tex && pdflatex main.tex
|
||||
```
|
||||
|
||||
If the unmodified template doesn't compile, fix that first (usually missing TeX packages — install via `tlmgr install <package>`).
|
||||
|
||||
**Step 3: Keep Template Content as Reference**
|
||||
|
||||
Don't immediately delete example content. Comment it out and use as formatting reference:
|
||||
```latex
|
||||
% Template example (keep for reference):
|
||||
% \begin{figure}[t]
|
||||
% \centering
|
||||
% \includegraphics[width=0.8\linewidth]{example-image}
|
||||
% \caption{Template shows caption style}
|
||||
% \end{figure}
|
||||
|
||||
% Your actual figure:
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.8\linewidth]{your-figure.pdf}
|
||||
\caption{Your caption following the same style.}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
**Step 4: Replace Content Section by Section**
|
||||
|
||||
Work through systematically: title/authors → abstract → introduction → methods → experiments → related work → conclusion → references → appendix. Compile after each section.
|
||||
|
||||
**Step 5: Use Template Macros**
|
||||
|
||||
```latex
|
||||
\newcommand{\method}{YourMethodName} % Consistent method naming
|
||||
\newcommand{\eg}{e.g.,\xspace} % Proper abbreviations
|
||||
\newcommand{\ie}{i.e.,\xspace}
|
||||
```
|
||||
|
||||
### Template Pitfalls
|
||||
|
||||
| Pitfall | Problem | Solution |
|
||||
|---------|---------|----------|
|
||||
| Copying only `.tex` file | Missing `.sty`, won't compile | Copy entire directory |
|
||||
| Modifying `.sty` files | Breaks conference formatting | Never edit style files |
|
||||
| Adding random packages | Conflicts, breaks template | Only add if necessary |
|
||||
| Deleting template content early | Lose formatting reference | Keep as comments until done |
|
||||
| Not compiling frequently | Errors accumulate | Compile after each section |
|
||||
| Raster PNGs for figures | Blurry in paper | Always use vector PDF via `savefig('fig.pdf')` |
|
||||
|
||||
### Quick Template Reference
|
||||
|
||||
| Conference | Main File | Style File | Page Limit |
|
||||
|------------|-----------|------------|------------|
|
||||
| NeurIPS 2025 | `main.tex` | `neurips.sty` | 9 pages |
|
||||
| ICML 2026 | `example_paper.tex` | `icml2026.sty` | 8 pages |
|
||||
| ICLR 2026 | `iclr2026_conference.tex` | `iclr2026_conference.sty` | 9 pages |
|
||||
| ACL 2025 | `acl_latex.tex` | `acl.sty` | 8 pages (long) |
|
||||
| AAAI 2026 | `aaai2026-unified-template.tex` | `aaai2026.sty` | 7 pages |
|
||||
| COLM 2025 | `colm2025_conference.tex` | `colm2025_conference.sty` | 9 pages |
|
||||
|
||||
**Universal**: Double-blind, references don't count, appendices unlimited, LaTeX required.
|
||||
|
||||
Templates in `templates/` directory. See [templates/README.md](templates/README.md) for compilation setup (VS Code, CLI, Overleaf, other IDEs).
|
||||
|
||||
### Tables and Figures
|
||||
|
||||
**Tables** — use `booktabs` for professional formatting:
|
||||
|
||||
```latex
|
||||
\usepackage{booktabs}
|
||||
\begin{tabular}{lcc}
|
||||
\toprule
|
||||
Method & Accuracy $\uparrow$ & Latency $\downarrow$ \\
|
||||
\midrule
|
||||
Baseline & 85.2 & 45ms \\
|
||||
\textbf{Ours} & \textbf{92.1} & 38ms \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Bold best value per metric
|
||||
- Include direction symbols ($\uparrow$ higher better, $\downarrow$ lower better)
|
||||
- Right-align numerical columns
|
||||
- Consistent decimal precision
|
||||
|
||||
**Figures**:
|
||||
- **Vector graphics** (PDF, EPS) for all plots and diagrams — `plt.savefig('fig.pdf')`
|
||||
- **Raster** (PNG 600 DPI) only for photographs
|
||||
- **Colorblind-safe palettes** (Okabe-Ito or Paul Tol)
|
||||
- Verify **grayscale readability** (8% of men have color vision deficiency)
|
||||
- **No title inside figure** — the caption serves this function
|
||||
- **Self-contained captions** — reader should understand without main text
|
||||
|
||||
### Conference Resubmission
|
||||
|
||||
For converting between venues, see Phase 7 (Submission Preparation) — it covers the full conversion workflow, page-change table, and post-rejection guidance.
|
||||
|
||||
### Professional LaTeX Preamble
|
||||
|
||||
Add these packages to any paper for professional quality. They are compatible with all major conference style files:
|
||||
|
||||
```latex
|
||||
% --- Professional Packages (add after conference style file) ---
|
||||
|
||||
% Typography
|
||||
\usepackage{microtype} % Microtypographic improvements (protrusion, expansion)
|
||||
% Makes text noticeably more polished — always include
|
||||
|
||||
% Tables
|
||||
\usepackage{booktabs} % Professional table rules (\toprule, \midrule, \bottomrule)
|
||||
\usepackage{siunitx} % Consistent number formatting, decimal alignment
|
||||
% Usage: \num{12345} → 12,345; \SI{3.5}{GHz} → 3.5 GHz
|
||||
% Table alignment: S column type for decimal-aligned numbers
|
||||
|
||||
% Figures
|
||||
\usepackage{graphicx} % Include graphics (\includegraphics)
|
||||
\usepackage{subcaption} % Subfigures with (a), (b), (c) labels
|
||||
% Usage: \begin{subfigure}{0.48\textwidth} ... \end{subfigure}
|
||||
|
||||
% Diagrams and Algorithms
|
||||
\usepackage{tikz} % Programmable vector diagrams
|
||||
\usetikzlibrary{arrows.meta, positioning, shapes.geometric, calc, fit, backgrounds}
|
||||
\usepackage[ruled,vlined]{algorithm2e} % Professional pseudocode
|
||||
% Alternative: \usepackage{algorithmicx} if template bundles it
|
||||
|
||||
% Cross-references
|
||||
\usepackage{cleveref} % Smart references: \cref{fig:x} → "Figure 1"
|
||||
% MUST be loaded AFTER hyperref
|
||||
% Handles: figures, tables, sections, equations, algorithms
|
||||
|
||||
% Math (usually included by conference .sty, but verify)
|
||||
\usepackage{amsmath,amssymb} % AMS math environments and symbols
|
||||
\usepackage{mathtools} % Extends amsmath (dcases, coloneqq, etc.)
|
||||
|
||||
% Colors (for figures and diagrams)
|
||||
\usepackage{xcolor} % Color management
|
||||
% Okabe-Ito colorblind-safe palette:
|
||||
\definecolor{okblue}{HTML}{0072B2}
|
||||
\definecolor{okorange}{HTML}{E69F00}
|
||||
\definecolor{okgreen}{HTML}{009E73}
|
||||
\definecolor{okred}{HTML}{D55E00}
|
||||
\definecolor{okpurple}{HTML}{CC79A7}
|
||||
\definecolor{okcyan}{HTML}{56B4E9}
|
||||
\definecolor{okyellow}{HTML}{F0E442}
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
- `microtype` is the single highest-impact package for visual quality. It adjusts character spacing at a sub-pixel level. Always include it.
|
||||
- `siunitx` handles decimal alignment in tables via the `S` column type — eliminates manual spacing.
|
||||
- `cleveref` must be loaded **after** `hyperref`. Most conference .sty files load hyperref, so put cleveref last.
|
||||
- Check if the conference template already loads any of these (especially `algorithm`, `amsmath`, `graphicx`). Don't double-load.
|
||||
|
||||
### siunitx Table Alignment
|
||||
|
||||
`siunitx` makes number-heavy tables significantly more readable:
|
||||
|
||||
```latex
|
||||
\begin{tabular}{l S[table-format=2.1] S[table-format=2.1] S[table-format=2.1]}
|
||||
\toprule
|
||||
Method & {Accuracy $\uparrow$} & {F1 $\uparrow$} & {Latency (ms) $\downarrow$} \\
|
||||
\midrule
|
||||
Baseline & 85.2 & 83.7 & 45.3 \\
|
||||
Ablation (no X) & 87.1 & 85.4 & 42.1 \\
|
||||
\textbf{Ours} & \textbf{92.1} & \textbf{90.8} & \textbf{38.7} \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
```
|
||||
|
||||
The `S` column type auto-aligns on the decimal point. Headers in `{}` escape the alignment.
|
||||
|
||||
### Subfigures
|
||||
|
||||
Standard pattern for side-by-side figures:
|
||||
|
||||
```latex
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\begin{subfigure}[b]{0.48\textwidth}
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{fig_results_a.pdf}
|
||||
\caption{Results on Dataset A.}
|
||||
\label{fig:results-a}
|
||||
\end{subfigure}
|
||||
\hfill
|
||||
\begin{subfigure}[b]{0.48\textwidth}
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{fig_results_b.pdf}
|
||||
\caption{Results on Dataset B.}
|
||||
\label{fig:results-b}
|
||||
\end{subfigure}
|
||||
\caption{Comparison of our method across two datasets. (a) shows the scaling
|
||||
behavior and (b) shows the ablation results. Both use 5 random seeds.}
|
||||
\label{fig:results}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
Use `\cref{fig:results}` → "Figure 1", `\cref{fig:results-a}` → "Figure 1a".
|
||||
|
||||
### Pseudocode with algorithm2e
|
||||
|
||||
```latex
|
||||
\begin{algorithm}[t]
|
||||
\caption{Iterative Refinement with Judge Panel}
|
||||
\label{alg:method}
|
||||
\KwIn{Task $T$, model $M$, judges $J_1 \ldots J_n$, convergence threshold $k$}
|
||||
\KwOut{Final output $A^*$}
|
||||
$A \gets M(T)$ \tcp*{Initial generation}
|
||||
$\text{streak} \gets 0$\;
|
||||
\While{$\text{streak} < k$}{
|
||||
$C \gets \text{Critic}(A, T)$ \tcp*{Identify weaknesses}
|
||||
$B \gets M(T, C)$ \tcp*{Revised version addressing critique}
|
||||
$AB \gets \text{Synthesize}(A, B)$ \tcp*{Merge best elements}
|
||||
\ForEach{judge $J_i$}{
|
||||
$\text{rank}_i \gets J_i(\text{shuffle}(A, B, AB))$ \tcp*{Blind ranking}
|
||||
}
|
||||
$\text{winner} \gets \text{BordaCount}(\text{ranks})$\;
|
||||
\eIf{$\text{winner} = A$}{
|
||||
$\text{streak} \gets \text{streak} + 1$\;
|
||||
}{
|
||||
$A \gets \text{winner}$; $\text{streak} \gets 0$\;
|
||||
}
|
||||
}
|
||||
\Return{$A$}\;
|
||||
\end{algorithm}
|
||||
```
|
||||
|
||||
### TikZ Diagram Patterns
|
||||
|
||||
TikZ is the standard for method diagrams in ML papers. Common patterns:
|
||||
|
||||
**Pipeline/Flow Diagram** (most common in ML papers):
|
||||
|
||||
```latex
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\begin{tikzpicture}[
|
||||
node distance=1.8cm,
|
||||
box/.style={rectangle, draw, rounded corners, minimum height=1cm,
|
||||
minimum width=2cm, align=center, font=\small},
|
||||
arrow/.style={-{Stealth[length=3mm]}, thick},
|
||||
]
|
||||
\node[box, fill=okcyan!20] (input) {Input\\$x$};
|
||||
\node[box, fill=okblue!20, right of=input] (encoder) {Encoder\\$f_\theta$};
|
||||
\node[box, fill=okgreen!20, right of=encoder] (latent) {Latent\\$z$};
|
||||
\node[box, fill=okorange!20, right of=latent] (decoder) {Decoder\\$g_\phi$};
|
||||
\node[box, fill=okred!20, right of=decoder] (output) {Output\\$\hat{x}$};
|
||||
|
||||
\draw[arrow] (input) -- (encoder);
|
||||
\draw[arrow] (encoder) -- (latent);
|
||||
\draw[arrow] (latent) -- (decoder);
|
||||
\draw[arrow] (decoder) -- (output);
|
||||
\end{tikzpicture}
|
||||
\caption{Architecture overview. The encoder maps input $x$ to latent
|
||||
representation $z$, which the decoder reconstructs.}
|
||||
\label{fig:architecture}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
**Comparison/Matrix Diagram** (for showing method variants):
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}[
|
||||
cell/.style={rectangle, draw, minimum width=2.5cm, minimum height=1cm,
|
||||
align=center, font=\small},
|
||||
header/.style={cell, fill=gray!20, font=\small\bfseries},
|
||||
]
|
||||
% Headers
|
||||
\node[header] at (0, 0) {Method};
|
||||
\node[header] at (3, 0) {Converges?};
|
||||
\node[header] at (6, 0) {Quality?};
|
||||
% Rows
|
||||
\node[cell] at (0, -1) {Single Pass};
|
||||
\node[cell, fill=okgreen!15] at (3, -1) {N/A};
|
||||
\node[cell, fill=okorange!15] at (6, -1) {Baseline};
|
||||
\node[cell] at (0, -2) {Critique+Revise};
|
||||
\node[cell, fill=okred!15] at (3, -2) {No};
|
||||
\node[cell, fill=okred!15] at (6, -2) {Degrades};
|
||||
\node[cell] at (0, -3) {Ours};
|
||||
\node[cell, fill=okgreen!15] at (3, -3) {Yes ($k$=2)};
|
||||
\node[cell, fill=okgreen!15] at (6, -3) {Improves};
|
||||
\end{tikzpicture}
|
||||
```
|
||||
|
||||
**Iterative Loop Diagram** (for methods with feedback):
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}[
|
||||
node distance=2cm,
|
||||
box/.style={rectangle, draw, rounded corners, minimum height=0.8cm,
|
||||
minimum width=1.8cm, align=center, font=\small},
|
||||
arrow/.style={-{Stealth[length=3mm]}, thick},
|
||||
label/.style={font=\scriptsize, midway, above},
|
||||
]
|
||||
\node[box, fill=okblue!20] (gen) {Generator};
|
||||
\node[box, fill=okred!20, right=2.5cm of gen] (critic) {Critic};
|
||||
\node[box, fill=okgreen!20, below=1.5cm of $(gen)!0.5!(critic)$] (judge) {Judge Panel};
|
||||
|
||||
\draw[arrow] (gen) -- node[label] {output $A$} (critic);
|
||||
\draw[arrow] (critic) -- node[label, right] {critique $C$} (judge);
|
||||
\draw[arrow] (judge) -| node[label, left, pos=0.3] {winner} (gen);
|
||||
\end{tikzpicture}
|
||||
```
|
||||
|
||||
### latexdiff for Revision Tracking
|
||||
|
||||
Essential for rebuttals — generates a marked-up PDF showing changes between versions:
|
||||
|
||||
```bash
|
||||
# Install
|
||||
# macOS: brew install latexdiff (or comes with TeX Live)
|
||||
# Linux: sudo apt install latexdiff
|
||||
|
||||
# Generate diff
|
||||
latexdiff paper_v1.tex paper_v2.tex > paper_diff.tex
|
||||
pdflatex paper_diff.tex
|
||||
|
||||
# For multi-file projects (with \input{} or \include{})
|
||||
latexdiff --flatten paper_v1.tex paper_v2.tex > paper_diff.tex
|
||||
```
|
||||
|
||||
This produces a PDF with deletions in red strikethrough and additions in blue — standard format for rebuttal supplements.
|
||||
|
||||
### SciencePlots for matplotlib
|
||||
|
||||
Install and use for publication-quality plots:
|
||||
|
||||
```bash
|
||||
pip install SciencePlots
|
||||
```
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import scienceplots # registers styles
|
||||
|
||||
# Use science style (IEEE-like, clean)
|
||||
with plt.style.context(['science', 'no-latex']):
|
||||
fig, ax = plt.subplots(figsize=(3.5, 2.5)) # Single-column width
|
||||
ax.plot(x, y, label='Ours', color='#0072B2')
|
||||
ax.plot(x, y2, label='Baseline', color='#D55E00', linestyle='--')
|
||||
ax.set_xlabel('Training Steps')
|
||||
ax.set_ylabel('Accuracy')
|
||||
ax.legend()
|
||||
fig.savefig('paper/fig_results.pdf', bbox_inches='tight')
|
||||
|
||||
# Available styles: 'science', 'ieee', 'nature', 'science+ieee'
|
||||
# Add 'no-latex' if LaTeX is not installed on the machine generating plots
|
||||
```
|
||||
|
||||
**Standard figure sizes** (two-column format):
|
||||
- Single column: `figsize=(3.5, 2.5)` — fits in one column
|
||||
- Double column: `figsize=(7.0, 3.0)` — spans both columns
|
||||
- Square: `figsize=(3.5, 3.5)` — for heatmaps, confusion matrices
|
||||
|
||||
---
|
||||
The complete drafting procedure (section-by-section order, LaTeX scaffolding, figure/table
|
||||
conventions, abstract and intro formulas, related-work positioning) lives in
|
||||
`references/phase5-paper-drafting.md` — load it with `read_file` when you reach this phase.
|
||||
Pair it with `references/writing-guide.md` for prose-level style rules.
|
||||
|
||||
## Phase 6: Self-Review & Revision
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,757 @@
|
|||
# Phase 5: Paper Drafting (full procedure)
|
||||
|
||||
**Goal**: Write a complete, publication-ready paper.
|
||||
|
||||
### Context Management for Large Projects
|
||||
|
||||
A paper project with 50+ experiment files, multiple result directories, and extensive literature notes can easily exceed the agent's context window. Manage this proactively:
|
||||
|
||||
**What to load into context per drafting task:**
|
||||
|
||||
| Drafting Task | Load Into Context | Do NOT Load |
|
||||
|---------------|------------------|-------------|
|
||||
| Writing Introduction | `experiment_log.md`, contribution statement, 5-10 most relevant paper abstracts | Raw result JSONs, full experiment scripts, all literature notes |
|
||||
| Writing Methods | Experiment configs, pseudocode, architecture description | Raw logs, results from other experiments |
|
||||
| Writing Results | `experiment_log.md`, result summary tables, figure list | Full analysis scripts, intermediate data |
|
||||
| Writing Related Work | Organized citation notes (Step 1.4 output), .bib file | Experiment files, raw PDFs |
|
||||
| Revision pass | Full paper draft, specific reviewer concerns | Everything else |
|
||||
|
||||
**Principles:**
|
||||
- **`experiment_log.md` is the primary context bridge** — it summarizes everything needed for writing without loading raw data files (see Step 4.6)
|
||||
- **Load one section's context at a time** when delegating. A sub-agent drafting Methods doesn't need the literature review notes.
|
||||
- **Summarize, don't include raw files.** For a 200-line result JSON, load a 10-line summary table. For a 50-page related paper, load the 5-sentence abstract + your 2-line note about its relevance.
|
||||
- **For very large projects**: Create a `context/` directory with pre-compressed summaries:
|
||||
```
|
||||
context/
|
||||
contribution.md # 1 sentence
|
||||
experiment_summary.md # Key results table (from experiment_log.md)
|
||||
literature_map.md # Organized citation notes
|
||||
figure_inventory.md # List of figures with descriptions
|
||||
```
|
||||
|
||||
### The Narrative Principle
|
||||
|
||||
**The single most critical insight**: Your paper is not a collection of experiments — it's a story with one clear contribution supported by evidence.
|
||||
|
||||
Every successful ML paper centers on what Neel Nanda calls "the narrative": a short, rigorous, evidence-based technical story with a takeaway readers care about.
|
||||
|
||||
**Three Pillars (must be crystal clear by end of introduction):**
|
||||
|
||||
| Pillar | Description | Test |
|
||||
|--------|-------------|------|
|
||||
| **The What** | 1-3 specific novel claims | Can you state them in one sentence? |
|
||||
| **The Why** | Rigorous empirical evidence | Do experiments distinguish your hypothesis from alternatives? |
|
||||
| **The So What** | Why readers should care | Does this connect to a recognized community problem? |
|
||||
|
||||
**If you cannot state your contribution in one sentence, you don't yet have a paper.**
|
||||
|
||||
### The Sources Behind This Guidance
|
||||
|
||||
This skill synthesizes writing philosophy from researchers who have published extensively at top venues. The writing philosophy layer was originally compiled by [Orchestra Research](https://github.com/orchestra-research) as the `ml-paper-writing` skill.
|
||||
|
||||
| Source | Key Contribution | Link |
|
||||
|--------|-----------------|------|
|
||||
| **Neel Nanda** (Google DeepMind) | The Narrative Principle, What/Why/So What framework | [How to Write ML Papers](https://www.alignmentforum.org/posts/eJGptPbbFPZGLpjsp/highly-opinionated-advice-on-how-to-write-ml-papers) |
|
||||
| **Sebastian Farquhar** (DeepMind) | 5-sentence abstract formula | [How to Write ML Papers](https://sebastianfarquhar.com/on-research/2024/11/04/how_to_write_ml_papers/) |
|
||||
| **Gopen & Swan** | 7 principles of reader expectations | [Science of Scientific Writing](https://cseweb.ucsd.edu/~swanson/papers/science-of-writing.pdf) |
|
||||
| **Zachary Lipton** | Word choice, eliminating hedging | [Heuristics for Scientific Writing](https://www.approximatelycorrect.com/2018/01/29/heuristics-technical-scientific-writing-machine-learning-perspective/) |
|
||||
| **Jacob Steinhardt** (UC Berkeley) | Precision, consistent terminology | [Writing Tips](https://bounded-regret.ghost.io/) |
|
||||
| **Ethan Perez** (Anthropic) | Micro-level clarity tips | [Easy Paper Writing Tips](https://ethanperez.net/easy-paper-writing-tips/) |
|
||||
| **Andrej Karpathy** | Single contribution focus | Various lectures |
|
||||
|
||||
**For deeper dives into any of these, see:**
|
||||
- [references/writing-guide.md](references/writing-guide.md) — Full explanations with examples
|
||||
- [references/sources.md](references/sources.md) — Complete bibliography
|
||||
|
||||
### Time Allocation
|
||||
|
||||
Spend approximately **equal time** on each of:
|
||||
1. The abstract
|
||||
2. The introduction
|
||||
3. The figures
|
||||
4. Everything else combined
|
||||
|
||||
**Why?** Most reviewers form judgments before reaching your methods. Readers encounter your paper as: title → abstract → introduction → figures → maybe the rest.
|
||||
|
||||
### Writing Workflow
|
||||
|
||||
```
|
||||
Paper Writing Checklist:
|
||||
- [ ] Step 1: Define the one-sentence contribution
|
||||
- [ ] Step 2: Draft Figure 1 (core idea or most compelling result)
|
||||
- [ ] Step 3: Draft abstract (5-sentence formula)
|
||||
- [ ] Step 4: Draft introduction (1-1.5 pages max)
|
||||
- [ ] Step 5: Draft methods
|
||||
- [ ] Step 6: Draft experiments & results
|
||||
- [ ] Step 7: Draft related work
|
||||
- [ ] Step 8: Draft conclusion & discussion
|
||||
- [ ] Step 9: Draft limitations (REQUIRED by all venues)
|
||||
- [ ] Step 10: Plan appendix (proofs, extra experiments, details)
|
||||
- [ ] Step 11: Complete paper checklist
|
||||
- [ ] Step 12: Final review
|
||||
```
|
||||
|
||||
### Two-Pass Refinement Pattern
|
||||
|
||||
When drafting with an AI agent, use a **two-pass** approach (proven effective in SakanaAI's AI-Scientist pipeline):
|
||||
|
||||
**Pass 1 — Write + immediate refine per section:**
|
||||
For each section, write a complete draft, then immediately refine it in the same context. This catches local issues (clarity, flow, completeness) while the section is fresh.
|
||||
|
||||
**Pass 2 — Global refinement with full-paper context:**
|
||||
After all sections are drafted, revisit each section with awareness of the complete paper. This catches cross-section issues: redundancy, inconsistent terminology, narrative flow, and gaps where one section promises something another doesn't deliver.
|
||||
|
||||
```
|
||||
Second-pass refinement prompt (per section):
|
||||
"Review the [SECTION] in the context of the complete paper.
|
||||
- Does it fit with the rest of the paper? Are there redundancies with other sections?
|
||||
- Is terminology consistent with Introduction and Methods?
|
||||
- Can anything be cut without weakening the message?
|
||||
- Does the narrative flow from the previous section and into the next?
|
||||
Make minimal, targeted edits. Do not rewrite from scratch."
|
||||
```
|
||||
|
||||
### LaTeX Error Checklist
|
||||
|
||||
Append this checklist to every refinement prompt. These are the most common errors when LLMs write LaTeX:
|
||||
|
||||
```
|
||||
LaTeX Quality Checklist (verify after every edit):
|
||||
- [ ] No unenclosed math symbols ($ signs balanced)
|
||||
- [ ] Only reference figures/tables that exist (\ref matches \label)
|
||||
- [ ] No fabricated citations (\cite matches entries in .bib)
|
||||
- [ ] Every \begin{env} has matching \end{env} (especially figure, table, algorithm)
|
||||
- [ ] No HTML contamination (</end{figure}> instead of \end{figure})
|
||||
- [ ] No unescaped underscores outside math mode (use \_ in text)
|
||||
- [ ] No duplicate \label definitions
|
||||
- [ ] No duplicate section headers
|
||||
- [ ] Numbers in text match actual experimental results
|
||||
- [ ] All figures have captions and labels
|
||||
- [ ] No overly long lines that cause overfull hbox warnings
|
||||
```
|
||||
|
||||
### Step 5.0: Title
|
||||
|
||||
The title is the single most-read element of the paper. It determines whether anyone clicks through to the abstract.
|
||||
|
||||
**Good titles**:
|
||||
- State the contribution or finding: "Autoreason: When Iterative LLM Refinement Works and Why It Fails"
|
||||
- Highlight a surprising result: "Scaling Data-Constrained Language Models" (implies you can)
|
||||
- Name the method + what it does: "DPO: Direct Preference Optimization of Language Models"
|
||||
|
||||
**Bad titles**:
|
||||
- Too generic: "An Approach to Improving Language Model Outputs"
|
||||
- Too long: anything over ~15 words
|
||||
- Jargon-only: "Asymptotic Convergence of Iterative Stochastic Policy Refinement" (who is this for?)
|
||||
|
||||
**Rules**:
|
||||
- Include your method name if you have one (for citability)
|
||||
- Include 1-2 keywords reviewers will search for
|
||||
- Avoid colons unless both halves carry meaning
|
||||
- Test: would a reviewer know the domain and contribution from the title alone?
|
||||
|
||||
### Step 5.1: Abstract (5-Sentence Formula)
|
||||
|
||||
From Sebastian Farquhar (DeepMind):
|
||||
|
||||
```
|
||||
1. What you achieved: "We introduce...", "We prove...", "We demonstrate..."
|
||||
2. Why this is hard and important
|
||||
3. How you do it (with specialist keywords for discoverability)
|
||||
4. What evidence you have
|
||||
5. Your most remarkable number/result
|
||||
```
|
||||
|
||||
**Delete** generic openings like "Large language models have achieved remarkable success..."
|
||||
|
||||
### Step 5.2: Figure 1
|
||||
|
||||
Figure 1 is the second thing most readers look at (after abstract). Draft it before writing the introduction — it forces you to clarify the core idea.
|
||||
|
||||
| Figure 1 Type | When to Use | Example |
|
||||
|---------------|-------------|---------|
|
||||
| **Method diagram** | New architecture or pipeline | TikZ flowchart showing your system |
|
||||
| **Results teaser** | One compelling result tells the whole story | Bar chart: "Ours vs baselines" with clear gap |
|
||||
| **Problem illustration** | The problem is unintuitive | Before/after showing failure mode you fix |
|
||||
| **Conceptual diagram** | Abstract contribution needs visual grounding | 2x2 matrix of method properties |
|
||||
|
||||
**Rules**: Figure 1 must be understandable without reading any text. The caption alone should communicate the core idea. Use color purposefully — don't just decorate.
|
||||
|
||||
### Step 5.3: Introduction (1-1.5 pages max)
|
||||
|
||||
Must include:
|
||||
- Clear problem statement
|
||||
- Brief approach overview
|
||||
- 2-4 bullet contribution list (max 1-2 lines each in two-column format)
|
||||
- Methods should start by page 2-3
|
||||
|
||||
### Step 5.4: Methods
|
||||
|
||||
Enable reimplementation:
|
||||
- Conceptual outline or pseudocode
|
||||
- All hyperparameters listed
|
||||
- Architectural details sufficient for reproduction
|
||||
- Present final design decisions; ablations go in experiments
|
||||
|
||||
### Step 5.5: Experiments & Results
|
||||
|
||||
For each experiment, explicitly state:
|
||||
- **What claim it supports**
|
||||
- How it connects to main contribution
|
||||
- What to observe: "the blue line shows X, which demonstrates Y"
|
||||
|
||||
Requirements:
|
||||
- Error bars with methodology (std dev vs std error)
|
||||
- Hyperparameter search ranges
|
||||
- Compute infrastructure (GPU type, total hours)
|
||||
- Seed-setting methods
|
||||
|
||||
### Step 5.6: Related Work
|
||||
|
||||
Organize methodologically, not paper-by-paper. Cite generously — reviewers likely authored relevant papers.
|
||||
|
||||
### Step 5.7: Limitations (REQUIRED)
|
||||
|
||||
All major conferences require this. Honesty helps:
|
||||
- Reviewers are instructed not to penalize honest limitation acknowledgment
|
||||
- Pre-empt criticisms by identifying weaknesses first
|
||||
- Explain why limitations don't undermine core claims
|
||||
|
||||
### Step 5.8: Conclusion & Discussion
|
||||
|
||||
**Conclusion** (required, 0.5-1 page):
|
||||
- Restate the contribution in one sentence (different wording from abstract)
|
||||
- Summarize key findings (2-3 sentences, not a list)
|
||||
- Implications: what does this mean for the field?
|
||||
- Future work: 2-3 concrete next steps (not vague "we leave X for future work")
|
||||
|
||||
**Discussion** (optional, sometimes combined with conclusion):
|
||||
- Broader implications beyond immediate results
|
||||
- Connections to other subfields
|
||||
- Honest assessment of when the method does and doesn't work
|
||||
- Practical deployment considerations
|
||||
|
||||
**Do NOT** introduce new results or claims in the conclusion.
|
||||
|
||||
### Step 5.9: Appendix Strategy
|
||||
|
||||
Appendices are unlimited at all major venues and are essential for reproducibility. Structure:
|
||||
|
||||
| Appendix Section | What Goes Here |
|
||||
|-----------------|---------------|
|
||||
| **Proofs & Derivations** | Full proofs too long for main text. Main text can state theorems with "proof in Appendix A." |
|
||||
| **Additional Experiments** | Ablations, scaling curves, per-dataset breakdowns, hyperparameter sensitivity |
|
||||
| **Implementation Details** | Full hyperparameter tables, training details, hardware specs, random seeds |
|
||||
| **Dataset Documentation** | Data collection process, annotation guidelines, licensing, preprocessing |
|
||||
| **Prompts & Templates** | Exact prompts used (for LLM-based methods), evaluation templates |
|
||||
| **Human Evaluation** | Annotation interface screenshots, instructions given to annotators, IRB details |
|
||||
| **Additional Figures** | Per-task breakdowns, trajectory visualizations, failure case examples |
|
||||
|
||||
**Rules**:
|
||||
- The main paper must be self-contained — reviewers are not required to read appendices
|
||||
- Never put critical evidence only in the appendix
|
||||
- Cross-reference: "Full results in Table 5 (Appendix B)" not just "see appendix"
|
||||
- Use `\appendix` command, then `\section{A: Proofs}` etc.
|
||||
|
||||
### Page Budget Management
|
||||
|
||||
When over the page limit:
|
||||
|
||||
| Cut Strategy | Saves | Risk |
|
||||
|-------------|-------|------|
|
||||
| Move proofs to appendix | 0.5-2 pages | Low — standard practice |
|
||||
| Condense related work | 0.5-1 page | Medium — may miss key citations |
|
||||
| Combine tables with subfigures | 0.25-0.5 page | Low — often improves readability |
|
||||
| Use `\vspace{-Xpt}` sparingly | 0.1-0.3 page | Low if subtle, high if obvious |
|
||||
| Remove qualitative examples | 0.5-1 page | Medium — reviewers like examples |
|
||||
| Reduce figure sizes | 0.25-0.5 page | High — figures must remain readable |
|
||||
|
||||
**Do NOT**: reduce font size, change margins, remove required sections (limitations, broader impact), or use `\small`/`\footnotesize` for main text.
|
||||
|
||||
### Step 5.10: Ethics & Broader Impact Statement
|
||||
|
||||
Most venues now require or strongly encourage an ethics/broader impact statement. This is not boilerplate — reviewers read it and can flag ethics concerns that trigger desk rejection.
|
||||
|
||||
**What to include:**
|
||||
|
||||
| Component | Content | Required By |
|
||||
|-----------|---------|-------------|
|
||||
| **Positive societal impact** | How your work benefits society | NeurIPS, ICML |
|
||||
| **Potential negative impact** | Misuse risks, dual-use concerns, failure modes | NeurIPS, ICML |
|
||||
| **Fairness & bias** | Does your method/data have known biases? | All venues (implicitly) |
|
||||
| **Environmental impact** | Compute carbon footprint for large-scale training | ICML, increasingly NeurIPS |
|
||||
| **Privacy** | Does your work use or enable processing of personal data? | ACL, NeurIPS |
|
||||
| **LLM disclosure** | Was AI used in writing or experiments? | ICLR (mandatory), ACL |
|
||||
|
||||
**Writing the statement:**
|
||||
|
||||
```latex
|
||||
\section*{Broader Impact Statement}
|
||||
% NeurIPS/ICML: after conclusion, does not count toward page limit
|
||||
|
||||
% 1. Positive applications (1-2 sentences)
|
||||
This work enables [specific application] which may benefit [specific group].
|
||||
|
||||
% 2. Risks and mitigations (1-3 sentences, be specific)
|
||||
[Method/model] could potentially be misused for [specific risk]. We mitigate
|
||||
this by [specific mitigation, e.g., releasing only model weights above size X,
|
||||
including safety filters, documenting failure modes].
|
||||
|
||||
% 3. Limitations of impact claims (1 sentence)
|
||||
Our evaluation is limited to [specific domain]; broader deployment would
|
||||
require [specific additional work].
|
||||
```
|
||||
|
||||
**Common mistakes:**
|
||||
- Writing "we foresee no negative impacts" (almost never true — reviewers distrust this)
|
||||
- Being vague: "this could be misused" without specifying how
|
||||
- Ignoring compute costs for large-scale work
|
||||
- Forgetting to disclose LLM use at venues that require it
|
||||
|
||||
**Compute carbon footprint** (for training-heavy papers):
|
||||
```python
|
||||
# Estimate using ML CO2 Impact tool methodology
|
||||
gpu_hours = 1000 # total GPU hours
|
||||
gpu_tdp_watts = 400 # e.g., A100 = 400W
|
||||
pue = 1.1 # Power Usage Effectiveness (data center overhead)
|
||||
carbon_intensity = 0.429 # kg CO2/kWh (US average; varies by region)
|
||||
|
||||
energy_kwh = (gpu_hours * gpu_tdp_watts * pue) / 1000
|
||||
carbon_kg = energy_kwh * carbon_intensity
|
||||
print(f"Energy: {energy_kwh:.0f} kWh, Carbon: {carbon_kg:.0f} kg CO2eq")
|
||||
```
|
||||
|
||||
### Step 5.11: Datasheets & Model Cards (If Applicable)
|
||||
|
||||
If your paper introduces a **new dataset** or **releases a model**, include structured documentation. Reviewers increasingly expect this, and NeurIPS Datasets & Benchmarks track requires it.
|
||||
|
||||
**Datasheets for Datasets** (Gebru et al., 2021) — include in appendix:
|
||||
|
||||
```
|
||||
Dataset Documentation (Appendix):
|
||||
- Motivation: Why was this dataset created? What task does it support?
|
||||
- Composition: What are the instances? How many? What data types?
|
||||
- Collection: How was data collected? What was the source?
|
||||
- Preprocessing: What cleaning/filtering was applied?
|
||||
- Distribution: How is the dataset distributed? Under what license?
|
||||
- Maintenance: Who maintains it? How to report issues?
|
||||
- Ethical considerations: Contains personal data? Consent obtained?
|
||||
Potential for harm? Known biases?
|
||||
```
|
||||
|
||||
**Model Cards** (Mitchell et al., 2019) — include in appendix for model releases:
|
||||
|
||||
```
|
||||
Model Card (Appendix):
|
||||
- Model details: Architecture, training data, training procedure
|
||||
- Intended use: Primary use cases, out-of-scope uses
|
||||
- Metrics: Evaluation metrics and results on benchmarks
|
||||
- Ethical considerations: Known biases, fairness evaluations
|
||||
- Limitations: Known failure modes, domains where model underperforms
|
||||
```
|
||||
|
||||
### Writing Style
|
||||
|
||||
**Sentence-level clarity (Gopen & Swan's 7 Principles):**
|
||||
|
||||
| Principle | Rule |
|
||||
|-----------|------|
|
||||
| Subject-verb proximity | Keep subject and verb close |
|
||||
| Stress position | Place emphasis at sentence ends |
|
||||
| Topic position | Put context first, new info after |
|
||||
| Old before new | Familiar info → unfamiliar info |
|
||||
| One unit, one function | Each paragraph makes one point |
|
||||
| Action in verb | Use verbs, not nominalizations |
|
||||
| Context before new | Set stage before presenting |
|
||||
|
||||
**Word choice (Lipton, Steinhardt):**
|
||||
- Be specific: "accuracy" not "performance"
|
||||
- Eliminate hedging: drop "may" unless genuinely uncertain
|
||||
- Consistent terminology throughout
|
||||
- Avoid incremental vocabulary: "develop", not "combine"
|
||||
|
||||
**Full writing guide with examples**: See [references/writing-guide.md](references/writing-guide.md)
|
||||
|
||||
### Using LaTeX Templates
|
||||
|
||||
**Always copy the entire template directory first, then write within it.**
|
||||
|
||||
```
|
||||
Template Setup Checklist:
|
||||
- [ ] Step 1: Copy entire template directory to new project
|
||||
- [ ] Step 2: Verify template compiles as-is (before any changes)
|
||||
- [ ] Step 3: Read the template's example content to understand structure
|
||||
- [ ] Step 4: Replace example content section by section
|
||||
- [ ] Step 5: Use template macros (check preamble for \newcommand definitions)
|
||||
- [ ] Step 6: Clean up template artifacts only at the end
|
||||
```
|
||||
|
||||
**Step 1: Copy the Full Template**
|
||||
|
||||
```bash
|
||||
cp -r templates/neurips2025/ ~/papers/my-paper/
|
||||
cd ~/papers/my-paper/
|
||||
ls -la # Should see: main.tex, neurips.sty, Makefile, etc.
|
||||
```
|
||||
|
||||
Copy the ENTIRE directory, not just the .tex file. Templates include style files (.sty), bibliography styles (.bst), example content, and Makefiles.
|
||||
|
||||
**Step 2: Verify Template Compiles First**
|
||||
|
||||
Before making ANY changes:
|
||||
```bash
|
||||
latexmk -pdf main.tex
|
||||
# Or manual: pdflatex main.tex && bibtex main && pdflatex main.tex && pdflatex main.tex
|
||||
```
|
||||
|
||||
If the unmodified template doesn't compile, fix that first (usually missing TeX packages — install via `tlmgr install <package>`).
|
||||
|
||||
**Step 3: Keep Template Content as Reference**
|
||||
|
||||
Don't immediately delete example content. Comment it out and use as formatting reference:
|
||||
```latex
|
||||
% Template example (keep for reference):
|
||||
% \begin{figure}[t]
|
||||
% \centering
|
||||
% \includegraphics[width=0.8\linewidth]{example-image}
|
||||
% \caption{Template shows caption style}
|
||||
% \end{figure}
|
||||
|
||||
% Your actual figure:
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.8\linewidth]{your-figure.pdf}
|
||||
\caption{Your caption following the same style.}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
**Step 4: Replace Content Section by Section**
|
||||
|
||||
Work through systematically: title/authors → abstract → introduction → methods → experiments → related work → conclusion → references → appendix. Compile after each section.
|
||||
|
||||
**Step 5: Use Template Macros**
|
||||
|
||||
```latex
|
||||
\newcommand{\method}{YourMethodName} % Consistent method naming
|
||||
\newcommand{\eg}{e.g.,\xspace} % Proper abbreviations
|
||||
\newcommand{\ie}{i.e.,\xspace}
|
||||
```
|
||||
|
||||
### Template Pitfalls
|
||||
|
||||
| Pitfall | Problem | Solution |
|
||||
|---------|---------|----------|
|
||||
| Copying only `.tex` file | Missing `.sty`, won't compile | Copy entire directory |
|
||||
| Modifying `.sty` files | Breaks conference formatting | Never edit style files |
|
||||
| Adding random packages | Conflicts, breaks template | Only add if necessary |
|
||||
| Deleting template content early | Lose formatting reference | Keep as comments until done |
|
||||
| Not compiling frequently | Errors accumulate | Compile after each section |
|
||||
| Raster PNGs for figures | Blurry in paper | Always use vector PDF via `savefig('fig.pdf')` |
|
||||
|
||||
### Quick Template Reference
|
||||
|
||||
| Conference | Main File | Style File | Page Limit |
|
||||
|------------|-----------|------------|------------|
|
||||
| NeurIPS 2025 | `main.tex` | `neurips.sty` | 9 pages |
|
||||
| ICML 2026 | `example_paper.tex` | `icml2026.sty` | 8 pages |
|
||||
| ICLR 2026 | `iclr2026_conference.tex` | `iclr2026_conference.sty` | 9 pages |
|
||||
| ACL 2025 | `acl_latex.tex` | `acl.sty` | 8 pages (long) |
|
||||
| AAAI 2026 | `aaai2026-unified-template.tex` | `aaai2026.sty` | 7 pages |
|
||||
| COLM 2025 | `colm2025_conference.tex` | `colm2025_conference.sty` | 9 pages |
|
||||
|
||||
**Universal**: Double-blind, references don't count, appendices unlimited, LaTeX required.
|
||||
|
||||
Templates in `templates/` directory. See [templates/README.md](templates/README.md) for compilation setup (VS Code, CLI, Overleaf, other IDEs).
|
||||
|
||||
### Tables and Figures
|
||||
|
||||
**Tables** — use `booktabs` for professional formatting:
|
||||
|
||||
```latex
|
||||
\usepackage{booktabs}
|
||||
\begin{tabular}{lcc}
|
||||
\toprule
|
||||
Method & Accuracy $\uparrow$ & Latency $\downarrow$ \\
|
||||
\midrule
|
||||
Baseline & 85.2 & 45ms \\
|
||||
\textbf{Ours} & \textbf{92.1} & 38ms \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Bold best value per metric
|
||||
- Include direction symbols ($\uparrow$ higher better, $\downarrow$ lower better)
|
||||
- Right-align numerical columns
|
||||
- Consistent decimal precision
|
||||
|
||||
**Figures**:
|
||||
- **Vector graphics** (PDF, EPS) for all plots and diagrams — `plt.savefig('fig.pdf')`
|
||||
- **Raster** (PNG 600 DPI) only for photographs
|
||||
- **Colorblind-safe palettes** (Okabe-Ito or Paul Tol)
|
||||
- Verify **grayscale readability** (8% of men have color vision deficiency)
|
||||
- **No title inside figure** — the caption serves this function
|
||||
- **Self-contained captions** — reader should understand without main text
|
||||
|
||||
### Conference Resubmission
|
||||
|
||||
For converting between venues, see Phase 7 (Submission Preparation) — it covers the full conversion workflow, page-change table, and post-rejection guidance.
|
||||
|
||||
### Professional LaTeX Preamble
|
||||
|
||||
Add these packages to any paper for professional quality. They are compatible with all major conference style files:
|
||||
|
||||
```latex
|
||||
% --- Professional Packages (add after conference style file) ---
|
||||
|
||||
% Typography
|
||||
\usepackage{microtype} % Microtypographic improvements (protrusion, expansion)
|
||||
% Makes text noticeably more polished — always include
|
||||
|
||||
% Tables
|
||||
\usepackage{booktabs} % Professional table rules (\toprule, \midrule, \bottomrule)
|
||||
\usepackage{siunitx} % Consistent number formatting, decimal alignment
|
||||
% Usage: \num{12345} → 12,345; \SI{3.5}{GHz} → 3.5 GHz
|
||||
% Table alignment: S column type for decimal-aligned numbers
|
||||
|
||||
% Figures
|
||||
\usepackage{graphicx} % Include graphics (\includegraphics)
|
||||
\usepackage{subcaption} % Subfigures with (a), (b), (c) labels
|
||||
% Usage: \begin{subfigure}{0.48\textwidth} ... \end{subfigure}
|
||||
|
||||
% Diagrams and Algorithms
|
||||
\usepackage{tikz} % Programmable vector diagrams
|
||||
\usetikzlibrary{arrows.meta, positioning, shapes.geometric, calc, fit, backgrounds}
|
||||
\usepackage[ruled,vlined]{algorithm2e} % Professional pseudocode
|
||||
% Alternative: \usepackage{algorithmicx} if template bundles it
|
||||
|
||||
% Cross-references
|
||||
\usepackage{cleveref} % Smart references: \cref{fig:x} → "Figure 1"
|
||||
% MUST be loaded AFTER hyperref
|
||||
% Handles: figures, tables, sections, equations, algorithms
|
||||
|
||||
% Math (usually included by conference .sty, but verify)
|
||||
\usepackage{amsmath,amssymb} % AMS math environments and symbols
|
||||
\usepackage{mathtools} % Extends amsmath (dcases, coloneqq, etc.)
|
||||
|
||||
% Colors (for figures and diagrams)
|
||||
\usepackage{xcolor} % Color management
|
||||
% Okabe-Ito colorblind-safe palette:
|
||||
\definecolor{okblue}{HTML}{0072B2}
|
||||
\definecolor{okorange}{HTML}{E69F00}
|
||||
\definecolor{okgreen}{HTML}{009E73}
|
||||
\definecolor{okred}{HTML}{D55E00}
|
||||
\definecolor{okpurple}{HTML}{CC79A7}
|
||||
\definecolor{okcyan}{HTML}{56B4E9}
|
||||
\definecolor{okyellow}{HTML}{F0E442}
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
- `microtype` is the single highest-impact package for visual quality. It adjusts character spacing at a sub-pixel level. Always include it.
|
||||
- `siunitx` handles decimal alignment in tables via the `S` column type — eliminates manual spacing.
|
||||
- `cleveref` must be loaded **after** `hyperref`. Most conference .sty files load hyperref, so put cleveref last.
|
||||
- Check if the conference template already loads any of these (especially `algorithm`, `amsmath`, `graphicx`). Don't double-load.
|
||||
|
||||
### siunitx Table Alignment
|
||||
|
||||
`siunitx` makes number-heavy tables significantly more readable:
|
||||
|
||||
```latex
|
||||
\begin{tabular}{l S[table-format=2.1] S[table-format=2.1] S[table-format=2.1]}
|
||||
\toprule
|
||||
Method & {Accuracy $\uparrow$} & {F1 $\uparrow$} & {Latency (ms) $\downarrow$} \\
|
||||
\midrule
|
||||
Baseline & 85.2 & 83.7 & 45.3 \\
|
||||
Ablation (no X) & 87.1 & 85.4 & 42.1 \\
|
||||
\textbf{Ours} & \textbf{92.1} & \textbf{90.8} & \textbf{38.7} \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
```
|
||||
|
||||
The `S` column type auto-aligns on the decimal point. Headers in `{}` escape the alignment.
|
||||
|
||||
### Subfigures
|
||||
|
||||
Standard pattern for side-by-side figures:
|
||||
|
||||
```latex
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\begin{subfigure}[b]{0.48\textwidth}
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{fig_results_a.pdf}
|
||||
\caption{Results on Dataset A.}
|
||||
\label{fig:results-a}
|
||||
\end{subfigure}
|
||||
\hfill
|
||||
\begin{subfigure}[b]{0.48\textwidth}
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{fig_results_b.pdf}
|
||||
\caption{Results on Dataset B.}
|
||||
\label{fig:results-b}
|
||||
\end{subfigure}
|
||||
\caption{Comparison of our method across two datasets. (a) shows the scaling
|
||||
behavior and (b) shows the ablation results. Both use 5 random seeds.}
|
||||
\label{fig:results}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
Use `\cref{fig:results}` → "Figure 1", `\cref{fig:results-a}` → "Figure 1a".
|
||||
|
||||
### Pseudocode with algorithm2e
|
||||
|
||||
```latex
|
||||
\begin{algorithm}[t]
|
||||
\caption{Iterative Refinement with Judge Panel}
|
||||
\label{alg:method}
|
||||
\KwIn{Task $T$, model $M$, judges $J_1 \ldots J_n$, convergence threshold $k$}
|
||||
\KwOut{Final output $A^*$}
|
||||
$A \gets M(T)$ \tcp*{Initial generation}
|
||||
$\text{streak} \gets 0$\;
|
||||
\While{$\text{streak} < k$}{
|
||||
$C \gets \text{Critic}(A, T)$ \tcp*{Identify weaknesses}
|
||||
$B \gets M(T, C)$ \tcp*{Revised version addressing critique}
|
||||
$AB \gets \text{Synthesize}(A, B)$ \tcp*{Merge best elements}
|
||||
\ForEach{judge $J_i$}{
|
||||
$\text{rank}_i \gets J_i(\text{shuffle}(A, B, AB))$ \tcp*{Blind ranking}
|
||||
}
|
||||
$\text{winner} \gets \text{BordaCount}(\text{ranks})$\;
|
||||
\eIf{$\text{winner} = A$}{
|
||||
$\text{streak} \gets \text{streak} + 1$\;
|
||||
}{
|
||||
$A \gets \text{winner}$; $\text{streak} \gets 0$\;
|
||||
}
|
||||
}
|
||||
\Return{$A$}\;
|
||||
\end{algorithm}
|
||||
```
|
||||
|
||||
### TikZ Diagram Patterns
|
||||
|
||||
TikZ is the standard for method diagrams in ML papers. Common patterns:
|
||||
|
||||
**Pipeline/Flow Diagram** (most common in ML papers):
|
||||
|
||||
```latex
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\begin{tikzpicture}[
|
||||
node distance=1.8cm,
|
||||
box/.style={rectangle, draw, rounded corners, minimum height=1cm,
|
||||
minimum width=2cm, align=center, font=\small},
|
||||
arrow/.style={-{Stealth[length=3mm]}, thick},
|
||||
]
|
||||
\node[box, fill=okcyan!20] (input) {Input\\$x$};
|
||||
\node[box, fill=okblue!20, right of=input] (encoder) {Encoder\\$f_\theta$};
|
||||
\node[box, fill=okgreen!20, right of=encoder] (latent) {Latent\\$z$};
|
||||
\node[box, fill=okorange!20, right of=latent] (decoder) {Decoder\\$g_\phi$};
|
||||
\node[box, fill=okred!20, right of=decoder] (output) {Output\\$\hat{x}$};
|
||||
|
||||
\draw[arrow] (input) -- (encoder);
|
||||
\draw[arrow] (encoder) -- (latent);
|
||||
\draw[arrow] (latent) -- (decoder);
|
||||
\draw[arrow] (decoder) -- (output);
|
||||
\end{tikzpicture}
|
||||
\caption{Architecture overview. The encoder maps input $x$ to latent
|
||||
representation $z$, which the decoder reconstructs.}
|
||||
\label{fig:architecture}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
**Comparison/Matrix Diagram** (for showing method variants):
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}[
|
||||
cell/.style={rectangle, draw, minimum width=2.5cm, minimum height=1cm,
|
||||
align=center, font=\small},
|
||||
header/.style={cell, fill=gray!20, font=\small\bfseries},
|
||||
]
|
||||
% Headers
|
||||
\node[header] at (0, 0) {Method};
|
||||
\node[header] at (3, 0) {Converges?};
|
||||
\node[header] at (6, 0) {Quality?};
|
||||
% Rows
|
||||
\node[cell] at (0, -1) {Single Pass};
|
||||
\node[cell, fill=okgreen!15] at (3, -1) {N/A};
|
||||
\node[cell, fill=okorange!15] at (6, -1) {Baseline};
|
||||
\node[cell] at (0, -2) {Critique+Revise};
|
||||
\node[cell, fill=okred!15] at (3, -2) {No};
|
||||
\node[cell, fill=okred!15] at (6, -2) {Degrades};
|
||||
\node[cell] at (0, -3) {Ours};
|
||||
\node[cell, fill=okgreen!15] at (3, -3) {Yes ($k$=2)};
|
||||
\node[cell, fill=okgreen!15] at (6, -3) {Improves};
|
||||
\end{tikzpicture}
|
||||
```
|
||||
|
||||
**Iterative Loop Diagram** (for methods with feedback):
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}[
|
||||
node distance=2cm,
|
||||
box/.style={rectangle, draw, rounded corners, minimum height=0.8cm,
|
||||
minimum width=1.8cm, align=center, font=\small},
|
||||
arrow/.style={-{Stealth[length=3mm]}, thick},
|
||||
label/.style={font=\scriptsize, midway, above},
|
||||
]
|
||||
\node[box, fill=okblue!20] (gen) {Generator};
|
||||
\node[box, fill=okred!20, right=2.5cm of gen] (critic) {Critic};
|
||||
\node[box, fill=okgreen!20, below=1.5cm of $(gen)!0.5!(critic)$] (judge) {Judge Panel};
|
||||
|
||||
\draw[arrow] (gen) -- node[label] {output $A$} (critic);
|
||||
\draw[arrow] (critic) -- node[label, right] {critique $C$} (judge);
|
||||
\draw[arrow] (judge) -| node[label, left, pos=0.3] {winner} (gen);
|
||||
\end{tikzpicture}
|
||||
```
|
||||
|
||||
### latexdiff for Revision Tracking
|
||||
|
||||
Essential for rebuttals — generates a marked-up PDF showing changes between versions:
|
||||
|
||||
```bash
|
||||
# Install
|
||||
# macOS: brew install latexdiff (or comes with TeX Live)
|
||||
# Linux: sudo apt install latexdiff
|
||||
|
||||
# Generate diff
|
||||
latexdiff paper_v1.tex paper_v2.tex > paper_diff.tex
|
||||
pdflatex paper_diff.tex
|
||||
|
||||
# For multi-file projects (with \input{} or \include{})
|
||||
latexdiff --flatten paper_v1.tex paper_v2.tex > paper_diff.tex
|
||||
```
|
||||
|
||||
This produces a PDF with deletions in red strikethrough and additions in blue — standard format for rebuttal supplements.
|
||||
|
||||
### SciencePlots for matplotlib
|
||||
|
||||
Install and use for publication-quality plots:
|
||||
|
||||
```bash
|
||||
pip install SciencePlots
|
||||
```
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import scienceplots # registers styles
|
||||
|
||||
# Use science style (IEEE-like, clean)
|
||||
with plt.style.context(['science', 'no-latex']):
|
||||
fig, ax = plt.subplots(figsize=(3.5, 2.5)) # Single-column width
|
||||
ax.plot(x, y, label='Ours', color='#0072B2')
|
||||
ax.plot(x, y2, label='Baseline', color='#D55E00', linestyle='--')
|
||||
ax.set_xlabel('Training Steps')
|
||||
ax.set_ylabel('Accuracy')
|
||||
ax.legend()
|
||||
fig.savefig('paper/fig_results.pdf', bbox_inches='tight')
|
||||
|
||||
# Available styles: 'science', 'ieee', 'nature', 'science+ieee'
|
||||
# Add 'no-latex' if LaTeX is not installed on the machine generating plots
|
||||
```
|
||||
|
||||
**Standard figure sizes** (two-column format):
|
||||
- Single column: `figsize=(3.5, 2.5)` — fits in one column
|
||||
- Double column: `figsize=(7.0, 3.0)` — spans both columns
|
||||
- Square: `figsize=(3.5, 3.5)` — for heatmaps, confusion matrices
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -0,0 +1,149 @@
|
|||
"""CI enforcement of the skill authoring standards (AGENTS.md hardline).
|
||||
|
||||
Every bundled (skills/) and optional (optional-skills/) SKILL.md must satisfy
|
||||
the programmatically-checkable subset of the authoring standards. Judgment
|
||||
calls (tier placement, router-skill smell, prose quality) stay with review;
|
||||
everything here is mechanical.
|
||||
|
||||
Pre-existing violations that need non-trivial content work are grandfathered
|
||||
in the GRANDFATHER dict below. Do NOT add new entries for new skills — fix
|
||||
the skill instead. Remove entries as the debt is paid down.
|
||||
"""
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
MARKETING = re.compile(
|
||||
r"\b(powerful|comprehensive|seamless|revolutionary|cutting-edge|state-of-the-art)\b",
|
||||
re.I,
|
||||
)
|
||||
MACHINE_LOCAL = re.compile(r"/home/(?!runner\b)[a-z0-9_-]+/|[A-Z]:\\+Users\\+(?!<)")
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Grandfathered pre-existing debt. Shrink this list; never grow it.
|
||||
# ---------------------------------------------------------------------------
|
||||
GRANDFATHER: dict[str, set[str]] = {
|
||||
# (empty — the Aug 2026 sweep cleared all mechanical violations)
|
||||
}
|
||||
|
||||
|
||||
def _skill_paths():
|
||||
return sorted(
|
||||
list(REPO.glob("skills/**/SKILL.md"))
|
||||
+ list(REPO.glob("optional-skills/**/SKILL.md"))
|
||||
)
|
||||
|
||||
|
||||
def _rel(p: Path) -> str:
|
||||
return str(p.parent.relative_to(REPO))
|
||||
|
||||
|
||||
def _params():
|
||||
return [pytest.param(p, id=_rel(p)) for p in _skill_paths()]
|
||||
|
||||
|
||||
def _grandfathered(p: Path, rule: str) -> bool:
|
||||
return rule in GRANDFATHER.get(_rel(p), set())
|
||||
|
||||
|
||||
def _frontmatter(p: Path):
|
||||
content = p.read_text(encoding="utf-8")
|
||||
assert content.startswith("---"), f"{_rel(p)}: SKILL.md must start with ---"
|
||||
m = re.search(r"\n---\s*\n", content[3:])
|
||||
assert m, f"{_rel(p)}: unclosed frontmatter"
|
||||
fm = yaml.safe_load(content[3 : m.start() + 3])
|
||||
assert isinstance(fm, dict), f"{_rel(p)}: frontmatter must be a YAML mapping"
|
||||
return fm, content
|
||||
|
||||
|
||||
ALL_SKILL_NAMES = None
|
||||
|
||||
|
||||
def _all_names():
|
||||
global ALL_SKILL_NAMES
|
||||
if ALL_SKILL_NAMES is None:
|
||||
names = set()
|
||||
for p in _skill_paths():
|
||||
names.add(p.parent.name)
|
||||
ALL_SKILL_NAMES = names
|
||||
return ALL_SKILL_NAMES
|
||||
|
||||
|
||||
def test_at_least_the_expected_population():
|
||||
# sanity: the globs actually find the trees (not a count snapshot)
|
||||
paths = _skill_paths()
|
||||
assert any("optional-skills" in str(p) for p in paths)
|
||||
assert any(str(p.parent).startswith(str(REPO / "skills")) for p in paths)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("p", _params())
|
||||
def test_required_frontmatter_fields(p):
|
||||
fm, _ = _frontmatter(p)
|
||||
missing = [
|
||||
f
|
||||
for f in ("name", "description", "version", "author", "license", "platforms")
|
||||
if f not in fm
|
||||
]
|
||||
if missing and not _grandfathered(p, "fields"):
|
||||
pytest.fail(f"{_rel(p)}: missing frontmatter fields: {missing}")
|
||||
hermes = (fm.get("metadata") or {}).get("hermes") or {}
|
||||
if not (hermes.get("tags") or fm.get("tags")) and not _grandfathered(p, "tags"):
|
||||
pytest.fail(f"{_rel(p)}: no tags (metadata.hermes.tags or top-level tags)")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("p", _params())
|
||||
def test_name_matches_directory(p):
|
||||
fm, _ = _frontmatter(p)
|
||||
if fm.get("name") != p.parent.name and not _grandfathered(p, "name"):
|
||||
pytest.fail(
|
||||
f"{_rel(p)}: frontmatter name {fm.get('name')!r} != dir {p.parent.name!r}"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("p", _params())
|
||||
def test_description_hardline(p):
|
||||
fm, _ = _frontmatter(p)
|
||||
desc = str(fm.get("description") or "")
|
||||
if _grandfathered(p, "description"):
|
||||
return
|
||||
assert len(desc) <= 60, f"{_rel(p)}: description {len(desc)} chars (hardline 60)"
|
||||
assert desc.rstrip().endswith("."), f"{_rel(p)}: description must end with a period"
|
||||
m = MARKETING.search(desc)
|
||||
assert not m, f"{_rel(p)}: marketing word in description: {m.group(0)!r}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("p", _params())
|
||||
def test_related_skills_resolve(p):
|
||||
fm, _ = _frontmatter(p)
|
||||
hermes = (fm.get("metadata") or {}).get("hermes") or {}
|
||||
dangling = [
|
||||
rs for rs in (hermes.get("related_skills") or []) if rs not in _all_names()
|
||||
]
|
||||
if dangling and not _grandfathered(p, "related"):
|
||||
pytest.fail(f"{_rel(p)}: dangling related_skills: {dangling}")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("p", _params())
|
||||
def test_no_machine_local_paths(p):
|
||||
_, content = _frontmatter(p)
|
||||
m = MACHINE_LOCAL.search(content)
|
||||
if m and not _grandfathered(p, "paths"):
|
||||
pytest.fail(f"{_rel(p)}: machine-local path {m.group(0)!r}")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("p", _params())
|
||||
def test_size_limit(p):
|
||||
_, content = _frontmatter(p)
|
||||
if len(content) > 100_000 and not _grandfathered(p, "size"):
|
||||
pytest.fail(
|
||||
f"{_rel(p)}: {len(content)} chars > 100k — split into references/"
|
||||
)
|
||||
|
||||
|
||||
def test_grandfather_entries_still_needed():
|
||||
"""A grandfather entry whose violation is fixed must be removed."""
|
||||
for rel in GRANDFATHER:
|
||||
assert (REPO / rel / "SKILL.md").exists(), f"stale grandfather entry: {rel}"
|
||||
|
|
@ -81,9 +81,9 @@ hermes skills uninstall <skill-name>
|
|||
| Skill | Description |
|
||||
|-------|-------------|
|
||||
| [**actual-setup**](/docs/user-guide/skills/optional/devops/devops-actual-setup) | Set up Actual Computer (actual.inc) inference in Hermes. |
|
||||
| [**inference-sh-cli**](/docs/user-guide/skills/optional/devops/devops-cli) | Run 150+ AI apps (image, video, LLM) via inference.sh CLI. |
|
||||
| [**docker-management**](/docs/user-guide/skills/optional/devops/devops-docker-management) | Manage Docker containers, images, volumes, and Compose. |
|
||||
| [**hermes-s6-container-supervision**](/docs/user-guide/skills/optional/devops/devops-hermes-s6-container-supervision) | Modify or debug s6 services in the Hermes Docker image. |
|
||||
| [**inference-sh-cli**](/docs/user-guide/skills/optional/devops/devops-inference-sh-cli) | Run 150+ AI apps (image, video, LLM) via inference.sh CLI. |
|
||||
| [**pinggy-tunnel**](/docs/user-guide/skills/optional/devops/devops-pinggy-tunnel) | Zero-install localhost tunnels over SSH via Pinggy. |
|
||||
| [**watchers**](/docs/user-guide/skills/optional/devops/devops-watchers) | Poll RSS, JSON APIs, and GitHub with watermark dedup. |
|
||||
|
||||
|
|
@ -145,35 +145,35 @@ hermes skills uninstall <skill-name>
|
|||
|
||||
| Skill | Description |
|
||||
|-------|-------------|
|
||||
| [**huggingface-accelerate**](/docs/user-guide/skills/optional/mlops/mlops-accelerate) | Run PyTorch training across GPUs with minimal changes. |
|
||||
| [**accelerate**](/docs/user-guide/skills/optional/mlops/mlops-accelerate) | Run PyTorch training across GPUs with minimal changes. |
|
||||
| [**axolotl**](/docs/user-guide/skills/optional/mlops/mlops-training-axolotl) | Axolotl: YAML LLM fine-tuning (LoRA, DPO, GRPO). |
|
||||
| [**chroma**](/docs/user-guide/skills/optional/mlops/mlops-chroma) | Embedding database for RAG and semantic search. |
|
||||
| [**clip**](/docs/user-guide/skills/optional/mlops/mlops-clip) | Zero-shot image classification and image-text search. |
|
||||
| [**dspy**](/docs/user-guide/skills/optional/mlops/mlops-research-dspy) | DSPy: declarative LM programs, auto-optimize prompts, RAG. |
|
||||
| [**faiss**](/docs/user-guide/skills/optional/mlops/mlops-faiss) | Fast vector similarity search at billion scale. |
|
||||
| [**optimizing-attention-flash**](/docs/user-guide/skills/optional/mlops/mlops-flash-attention) | Speed up long-sequence transformer training and inference. |
|
||||
| [**flash-attention**](/docs/user-guide/skills/optional/mlops/mlops-flash-attention) | Speed up long-sequence transformer training and inference. |
|
||||
| [**guidance**](/docs/user-guide/skills/optional/mlops/mlops-guidance) | Constrain LLM output with grammars; guarantee valid JSON. |
|
||||
| [**huggingface-tokenizers**](/docs/user-guide/skills/optional/mlops/mlops-huggingface-tokenizers) | Fast BPE/WordPiece tokenization and custom vocab training. |
|
||||
| [**instructor**](/docs/user-guide/skills/optional/mlops/mlops-instructor) | Structured LLM outputs validated with Pydantic. |
|
||||
| [**lambda-labs-gpu-cloud**](/docs/user-guide/skills/optional/mlops/mlops-lambda-labs) | On-demand GPU cloud instances for ML training. |
|
||||
| [**lambda-labs**](/docs/user-guide/skills/optional/mlops/mlops-lambda-labs) | On-demand GPU cloud instances for ML training. |
|
||||
| [**llava**](/docs/user-guide/skills/optional/mlops/mlops-llava) | Vision-language chat: VQA, captioning, image dialogue. |
|
||||
| [**modal-serverless-gpu**](/docs/user-guide/skills/optional/mlops/mlops-modal) | Serverless GPU cloud for ML jobs and model APIs. |
|
||||
| [**modal**](/docs/user-guide/skills/optional/mlops/mlops-modal) | Serverless GPU cloud for ML jobs and model APIs. |
|
||||
| [**nemo-curator**](/docs/user-guide/skills/optional/mlops/mlops-nemo-curator) | Curate LLM training data: dedupe, filter, PII redaction. |
|
||||
| [**obliteratus**](/docs/user-guide/skills/optional/mlops/mlops-obliteratus) | OBLITERATUS: abliterate LLM refusals (diff-in-means). |
|
||||
| [**outlines**](/docs/user-guide/skills/optional/mlops/mlops-inference-outlines) | Outlines: structured JSON/regex/Pydantic LLM generation. |
|
||||
| [**peft-fine-tuning**](/docs/user-guide/skills/optional/mlops/mlops-peft) | Fine-tune large LLMs with LoRA on limited GPU memory. |
|
||||
| [**peft**](/docs/user-guide/skills/optional/mlops/mlops-peft) | Fine-tune large LLMs with LoRA on limited GPU memory. |
|
||||
| [**pinecone**](/docs/user-guide/skills/optional/mlops/mlops-pinecone) | Managed vector DB for production RAG and search. |
|
||||
| [**pytorch-fsdp**](/docs/user-guide/skills/optional/mlops/mlops-pytorch-fsdp) | Fully sharded data-parallel training for large models. |
|
||||
| [**pytorch-lightning**](/docs/user-guide/skills/optional/mlops/mlops-pytorch-lightning) | Clean training loops with built-in distributed support. |
|
||||
| [**qdrant-vector-search**](/docs/user-guide/skills/optional/mlops/mlops-qdrant) | Vector search engine for production RAG systems. |
|
||||
| [**sparse-autoencoder-training**](/docs/user-guide/skills/optional/mlops/mlops-saelens) | Train sparse autoencoders to interpret model features. |
|
||||
| [**qdrant**](/docs/user-guide/skills/optional/mlops/mlops-qdrant) | Vector search engine for production RAG systems. |
|
||||
| [**saelens**](/docs/user-guide/skills/optional/mlops/mlops-saelens) | Train sparse autoencoders to interpret model features. |
|
||||
| [**segment-anything-model**](/docs/user-guide/skills/optional/mlops/mlops-models-segment-anything-model) | SAM: zero-shot image segmentation via points, boxes, masks. |
|
||||
| [**simpo-training**](/docs/user-guide/skills/optional/mlops/mlops-simpo) | Reference-free preference alignment, simpler than DPO. |
|
||||
| [**slime-rl-training**](/docs/user-guide/skills/optional/mlops/mlops-slime) | RL post-training for LLMs with Megatron and SGLang. |
|
||||
| [**stable-diffusion-image-generation**](/docs/user-guide/skills/optional/mlops/mlops-stable-diffusion) | Text-to-image generation, inpainting, and img2img. |
|
||||
| [**simpo**](/docs/user-guide/skills/optional/mlops/mlops-simpo) | Reference-free preference alignment, simpler than DPO. |
|
||||
| [**slime**](/docs/user-guide/skills/optional/mlops/mlops-slime) | RL post-training for LLMs with Megatron and SGLang. |
|
||||
| [**stable-diffusion**](/docs/user-guide/skills/optional/mlops/mlops-stable-diffusion) | Text-to-image generation, inpainting, and img2img. |
|
||||
| [**tensorrt-llm**](/docs/user-guide/skills/optional/mlops/mlops-tensorrt-llm) | High-throughput LLM inference on NVIDIA GPUs. |
|
||||
| [**distributed-llm-pretraining-torchtitan**](/docs/user-guide/skills/optional/mlops/mlops-torchtitan) | Pretrain LLMs at scale with PyTorch 4D parallelism. |
|
||||
| [**fine-tuning-with-trl**](/docs/user-guide/skills/optional/mlops/mlops-training-trl-fine-tuning) | TRL: SFT, DPO, GRPO, RLOO reward modeling for LLM RLHF. |
|
||||
| [**torchtitan**](/docs/user-guide/skills/optional/mlops/mlops-torchtitan) | Pretrain LLMs at scale with PyTorch 4D parallelism. |
|
||||
| [**trl-fine-tuning**](/docs/user-guide/skills/optional/mlops/mlops-training-trl-fine-tuning) | TRL: SFT, DPO, GRPO, RLOO reward modeling for LLM RLHF. |
|
||||
| [**unsloth**](/docs/user-guide/skills/optional/mlops/mlops-training-unsloth) | Unsloth: 2-5x faster LoRA/QLoRA fine-tuning, less VRAM. |
|
||||
| [**whisper**](/docs/user-guide/skills/optional/mlops/mlops-whisper) | Transcribe and translate speech in 99 languages. |
|
||||
|
||||
|
|
@ -190,7 +190,7 @@ hermes skills uninstall <skill-name>
|
|||
| Skill | Description |
|
||||
|-------|-------------|
|
||||
| [**canvas**](/docs/user-guide/skills/optional/productivity/productivity-canvas) | Fetch Canvas LMS courses and assignments via API token. |
|
||||
| [**here.now**](/docs/user-guide/skills/optional/productivity/productivity-here-now) | Publish sites to {slug}.here.now and store files in Drives. |
|
||||
| [**here-now**](/docs/user-guide/skills/optional/productivity/productivity-here-now) | Publish sites to {slug}.here.now and store files in Drives. |
|
||||
| [**memento-flashcards**](/docs/user-guide/skills/optional/productivity/productivity-memento-flashcards) | Spaced-repetition flashcards: create, review, quiz, export. |
|
||||
| [**shop**](/docs/user-guide/skills/optional/productivity/productivity-shop) | Shop catalog search, checkout, order tracking, returns. |
|
||||
| [**shopify**](/docs/user-guide/skills/optional/productivity/productivity-shopify) | Query Shopify Admin/Storefront GraphQL APIs via curl. |
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ Generate images, video, and audio via diffusion workflows.
|
|||
| License | MIT |
|
||||
| Platforms | macos, linux, windows |
|
||||
| Tags | `comfyui`, `image-generation`, `stable-diffusion`, `flux`, `sd3`, `wan-video`, `hunyuan-video`, `creative`, `generative-ai`, `video-generation` |
|
||||
| Related skills | [`stable-diffusion-image-generation`](/docs/user-guide/skills/optional/mlops/mlops-stable-diffusion) |
|
||||
| Related skills | [`stable-diffusion`](/docs/user-guide/skills/optional/mlops/mlops-stable-diffusion) |
|
||||
|
||||
## Reference: full SKILL.md
|
||||
|
||||
|
|
@ -328,7 +328,7 @@ For users without a capable GPU or who want zero setup. Hosted on RTX 6000 Pro.
|
|||
2. Generate an API key at https://platform.comfy.org/login
|
||||
3. Set the key:
|
||||
```bash
|
||||
export COMFY_CLOUD_API_KEY="comfyui-xxxxxxxxxxxx"
|
||||
export COMFY_CLOUD_API_KEY="your-comfyui-key"
|
||||
```
|
||||
4. Run workflows:
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -795,760 +795,10 @@ See [references/autoreason-methodology.md](https://github.com/NousResearch/herme
|
|||
|
||||
## Phase 5: Paper Drafting
|
||||
|
||||
**Goal**: Write a complete, publication-ready paper.
|
||||
|
||||
### Context Management for Large Projects
|
||||
|
||||
A paper project with 50+ experiment files, multiple result directories, and extensive literature notes can easily exceed the agent's context window. Manage this proactively:
|
||||
|
||||
**What to load into context per drafting task:**
|
||||
|
||||
| Drafting Task | Load Into Context | Do NOT Load |
|
||||
|---------------|------------------|-------------|
|
||||
| Writing Introduction | `experiment_log.md`, contribution statement, 5-10 most relevant paper abstracts | Raw result JSONs, full experiment scripts, all literature notes |
|
||||
| Writing Methods | Experiment configs, pseudocode, architecture description | Raw logs, results from other experiments |
|
||||
| Writing Results | `experiment_log.md`, result summary tables, figure list | Full analysis scripts, intermediate data |
|
||||
| Writing Related Work | Organized citation notes (Step 1.4 output), .bib file | Experiment files, raw PDFs |
|
||||
| Revision pass | Full paper draft, specific reviewer concerns | Everything else |
|
||||
|
||||
**Principles:**
|
||||
- **`experiment_log.md` is the primary context bridge** — it summarizes everything needed for writing without loading raw data files (see Step 4.6)
|
||||
- **Load one section's context at a time** when delegating. A sub-agent drafting Methods doesn't need the literature review notes.
|
||||
- **Summarize, don't include raw files.** For a 200-line result JSON, load a 10-line summary table. For a 50-page related paper, load the 5-sentence abstract + your 2-line note about its relevance.
|
||||
- **For very large projects**: Create a `context/` directory with pre-compressed summaries:
|
||||
```
|
||||
context/
|
||||
contribution.md # 1 sentence
|
||||
experiment_summary.md # Key results table (from experiment_log.md)
|
||||
literature_map.md # Organized citation notes
|
||||
figure_inventory.md # List of figures with descriptions
|
||||
```
|
||||
|
||||
### The Narrative Principle
|
||||
|
||||
**The single most critical insight**: Your paper is not a collection of experiments — it's a story with one clear contribution supported by evidence.
|
||||
|
||||
Every successful ML paper centers on what Neel Nanda calls "the narrative": a short, rigorous, evidence-based technical story with a takeaway readers care about.
|
||||
|
||||
**Three Pillars (must be crystal clear by end of introduction):**
|
||||
|
||||
| Pillar | Description | Test |
|
||||
|--------|-------------|------|
|
||||
| **The What** | 1-3 specific novel claims | Can you state them in one sentence? |
|
||||
| **The Why** | Rigorous empirical evidence | Do experiments distinguish your hypothesis from alternatives? |
|
||||
| **The So What** | Why readers should care | Does this connect to a recognized community problem? |
|
||||
|
||||
**If you cannot state your contribution in one sentence, you don't yet have a paper.**
|
||||
|
||||
### The Sources Behind This Guidance
|
||||
|
||||
This skill synthesizes writing philosophy from researchers who have published extensively at top venues. The writing philosophy layer was originally compiled by [Orchestra Research](https://github.com/orchestra-research) as the `ml-paper-writing` skill.
|
||||
|
||||
| Source | Key Contribution | Link |
|
||||
|--------|-----------------|------|
|
||||
| **Neel Nanda** (Google DeepMind) | The Narrative Principle, What/Why/So What framework | [How to Write ML Papers](https://www.alignmentforum.org/posts/eJGptPbbFPZGLpjsp/highly-opinionated-advice-on-how-to-write-ml-papers) |
|
||||
| **Sebastian Farquhar** (DeepMind) | 5-sentence abstract formula | [How to Write ML Papers](https://sebastianfarquhar.com/on-research/2024/11/04/how_to_write_ml_papers/) |
|
||||
| **Gopen & Swan** | 7 principles of reader expectations | [Science of Scientific Writing](https://cseweb.ucsd.edu/~swanson/papers/science-of-writing.pdf) |
|
||||
| **Zachary Lipton** | Word choice, eliminating hedging | [Heuristics for Scientific Writing](https://www.approximatelycorrect.com/2018/01/29/heuristics-technical-scientific-writing-machine-learning-perspective/) |
|
||||
| **Jacob Steinhardt** (UC Berkeley) | Precision, consistent terminology | [Writing Tips](https://bounded-regret.ghost.io/) |
|
||||
| **Ethan Perez** (Anthropic) | Micro-level clarity tips | [Easy Paper Writing Tips](https://ethanperez.net/easy-paper-writing-tips/) |
|
||||
| **Andrej Karpathy** | Single contribution focus | Various lectures |
|
||||
|
||||
**For deeper dives into any of these, see:**
|
||||
- [references/writing-guide.md](https://github.com/NousResearch/hermes-agent/blob/main/skills/research/research-paper-writing/references/writing-guide.md) — Full explanations with examples
|
||||
- [references/sources.md](https://github.com/NousResearch/hermes-agent/blob/main/skills/research/research-paper-writing/references/sources.md) — Complete bibliography
|
||||
|
||||
### Time Allocation
|
||||
|
||||
Spend approximately **equal time** on each of:
|
||||
1. The abstract
|
||||
2. The introduction
|
||||
3. The figures
|
||||
4. Everything else combined
|
||||
|
||||
**Why?** Most reviewers form judgments before reaching your methods. Readers encounter your paper as: title → abstract → introduction → figures → maybe the rest.
|
||||
|
||||
### Writing Workflow
|
||||
|
||||
```
|
||||
Paper Writing Checklist:
|
||||
- [ ] Step 1: Define the one-sentence contribution
|
||||
- [ ] Step 2: Draft Figure 1 (core idea or most compelling result)
|
||||
- [ ] Step 3: Draft abstract (5-sentence formula)
|
||||
- [ ] Step 4: Draft introduction (1-1.5 pages max)
|
||||
- [ ] Step 5: Draft methods
|
||||
- [ ] Step 6: Draft experiments & results
|
||||
- [ ] Step 7: Draft related work
|
||||
- [ ] Step 8: Draft conclusion & discussion
|
||||
- [ ] Step 9: Draft limitations (REQUIRED by all venues)
|
||||
- [ ] Step 10: Plan appendix (proofs, extra experiments, details)
|
||||
- [ ] Step 11: Complete paper checklist
|
||||
- [ ] Step 12: Final review
|
||||
```
|
||||
|
||||
### Two-Pass Refinement Pattern
|
||||
|
||||
When drafting with an AI agent, use a **two-pass** approach (proven effective in SakanaAI's AI-Scientist pipeline):
|
||||
|
||||
**Pass 1 — Write + immediate refine per section:**
|
||||
For each section, write a complete draft, then immediately refine it in the same context. This catches local issues (clarity, flow, completeness) while the section is fresh.
|
||||
|
||||
**Pass 2 — Global refinement with full-paper context:**
|
||||
After all sections are drafted, revisit each section with awareness of the complete paper. This catches cross-section issues: redundancy, inconsistent terminology, narrative flow, and gaps where one section promises something another doesn't deliver.
|
||||
|
||||
```
|
||||
Second-pass refinement prompt (per section):
|
||||
"Review the [SECTION] in the context of the complete paper.
|
||||
- Does it fit with the rest of the paper? Are there redundancies with other sections?
|
||||
- Is terminology consistent with Introduction and Methods?
|
||||
- Can anything be cut without weakening the message?
|
||||
- Does the narrative flow from the previous section and into the next?
|
||||
Make minimal, targeted edits. Do not rewrite from scratch."
|
||||
```
|
||||
|
||||
### LaTeX Error Checklist
|
||||
|
||||
Append this checklist to every refinement prompt. These are the most common errors when LLMs write LaTeX:
|
||||
|
||||
```
|
||||
LaTeX Quality Checklist (verify after every edit):
|
||||
- [ ] No unenclosed math symbols ($ signs balanced)
|
||||
- [ ] Only reference figures/tables that exist (\ref matches \label)
|
||||
- [ ] No fabricated citations (\cite matches entries in .bib)
|
||||
- [ ] Every \begin{env} has matching \end{env} (especially figure, table, algorithm)
|
||||
- [ ] No HTML contamination (</end{figure}> instead of \end{figure})
|
||||
- [ ] No unescaped underscores outside math mode (use \_ in text)
|
||||
- [ ] No duplicate \label definitions
|
||||
- [ ] No duplicate section headers
|
||||
- [ ] Numbers in text match actual experimental results
|
||||
- [ ] All figures have captions and labels
|
||||
- [ ] No overly long lines that cause overfull hbox warnings
|
||||
```
|
||||
|
||||
### Step 5.0: Title
|
||||
|
||||
The title is the single most-read element of the paper. It determines whether anyone clicks through to the abstract.
|
||||
|
||||
**Good titles**:
|
||||
- State the contribution or finding: "Autoreason: When Iterative LLM Refinement Works and Why It Fails"
|
||||
- Highlight a surprising result: "Scaling Data-Constrained Language Models" (implies you can)
|
||||
- Name the method + what it does: "DPO: Direct Preference Optimization of Language Models"
|
||||
|
||||
**Bad titles**:
|
||||
- Too generic: "An Approach to Improving Language Model Outputs"
|
||||
- Too long: anything over ~15 words
|
||||
- Jargon-only: "Asymptotic Convergence of Iterative Stochastic Policy Refinement" (who is this for?)
|
||||
|
||||
**Rules**:
|
||||
- Include your method name if you have one (for citability)
|
||||
- Include 1-2 keywords reviewers will search for
|
||||
- Avoid colons unless both halves carry meaning
|
||||
- Test: would a reviewer know the domain and contribution from the title alone?
|
||||
|
||||
### Step 5.1: Abstract (5-Sentence Formula)
|
||||
|
||||
From Sebastian Farquhar (DeepMind):
|
||||
|
||||
```
|
||||
1. What you achieved: "We introduce...", "We prove...", "We demonstrate..."
|
||||
2. Why this is hard and important
|
||||
3. How you do it (with specialist keywords for discoverability)
|
||||
4. What evidence you have
|
||||
5. Your most remarkable number/result
|
||||
```
|
||||
|
||||
**Delete** generic openings like "Large language models have achieved remarkable success..."
|
||||
|
||||
### Step 5.2: Figure 1
|
||||
|
||||
Figure 1 is the second thing most readers look at (after abstract). Draft it before writing the introduction — it forces you to clarify the core idea.
|
||||
|
||||
| Figure 1 Type | When to Use | Example |
|
||||
|---------------|-------------|---------|
|
||||
| **Method diagram** | New architecture or pipeline | TikZ flowchart showing your system |
|
||||
| **Results teaser** | One compelling result tells the whole story | Bar chart: "Ours vs baselines" with clear gap |
|
||||
| **Problem illustration** | The problem is unintuitive | Before/after showing failure mode you fix |
|
||||
| **Conceptual diagram** | Abstract contribution needs visual grounding | 2x2 matrix of method properties |
|
||||
|
||||
**Rules**: Figure 1 must be understandable without reading any text. The caption alone should communicate the core idea. Use color purposefully — don't just decorate.
|
||||
|
||||
### Step 5.3: Introduction (1-1.5 pages max)
|
||||
|
||||
Must include:
|
||||
- Clear problem statement
|
||||
- Brief approach overview
|
||||
- 2-4 bullet contribution list (max 1-2 lines each in two-column format)
|
||||
- Methods should start by page 2-3
|
||||
|
||||
### Step 5.4: Methods
|
||||
|
||||
Enable reimplementation:
|
||||
- Conceptual outline or pseudocode
|
||||
- All hyperparameters listed
|
||||
- Architectural details sufficient for reproduction
|
||||
- Present final design decisions; ablations go in experiments
|
||||
|
||||
### Step 5.5: Experiments & Results
|
||||
|
||||
For each experiment, explicitly state:
|
||||
- **What claim it supports**
|
||||
- How it connects to main contribution
|
||||
- What to observe: "the blue line shows X, which demonstrates Y"
|
||||
|
||||
Requirements:
|
||||
- Error bars with methodology (std dev vs std error)
|
||||
- Hyperparameter search ranges
|
||||
- Compute infrastructure (GPU type, total hours)
|
||||
- Seed-setting methods
|
||||
|
||||
### Step 5.6: Related Work
|
||||
|
||||
Organize methodologically, not paper-by-paper. Cite generously — reviewers likely authored relevant papers.
|
||||
|
||||
### Step 5.7: Limitations (REQUIRED)
|
||||
|
||||
All major conferences require this. Honesty helps:
|
||||
- Reviewers are instructed not to penalize honest limitation acknowledgment
|
||||
- Pre-empt criticisms by identifying weaknesses first
|
||||
- Explain why limitations don't undermine core claims
|
||||
|
||||
### Step 5.8: Conclusion & Discussion
|
||||
|
||||
**Conclusion** (required, 0.5-1 page):
|
||||
- Restate the contribution in one sentence (different wording from abstract)
|
||||
- Summarize key findings (2-3 sentences, not a list)
|
||||
- Implications: what does this mean for the field?
|
||||
- Future work: 2-3 concrete next steps (not vague "we leave X for future work")
|
||||
|
||||
**Discussion** (optional, sometimes combined with conclusion):
|
||||
- Broader implications beyond immediate results
|
||||
- Connections to other subfields
|
||||
- Honest assessment of when the method does and doesn't work
|
||||
- Practical deployment considerations
|
||||
|
||||
**Do NOT** introduce new results or claims in the conclusion.
|
||||
|
||||
### Step 5.9: Appendix Strategy
|
||||
|
||||
Appendices are unlimited at all major venues and are essential for reproducibility. Structure:
|
||||
|
||||
| Appendix Section | What Goes Here |
|
||||
|-----------------|---------------|
|
||||
| **Proofs & Derivations** | Full proofs too long for main text. Main text can state theorems with "proof in Appendix A." |
|
||||
| **Additional Experiments** | Ablations, scaling curves, per-dataset breakdowns, hyperparameter sensitivity |
|
||||
| **Implementation Details** | Full hyperparameter tables, training details, hardware specs, random seeds |
|
||||
| **Dataset Documentation** | Data collection process, annotation guidelines, licensing, preprocessing |
|
||||
| **Prompts & Templates** | Exact prompts used (for LLM-based methods), evaluation templates |
|
||||
| **Human Evaluation** | Annotation interface screenshots, instructions given to annotators, IRB details |
|
||||
| **Additional Figures** | Per-task breakdowns, trajectory visualizations, failure case examples |
|
||||
|
||||
**Rules**:
|
||||
- The main paper must be self-contained — reviewers are not required to read appendices
|
||||
- Never put critical evidence only in the appendix
|
||||
- Cross-reference: "Full results in Table 5 (Appendix B)" not just "see appendix"
|
||||
- Use `\appendix` command, then `\section{A: Proofs}` etc.
|
||||
|
||||
### Page Budget Management
|
||||
|
||||
When over the page limit:
|
||||
|
||||
| Cut Strategy | Saves | Risk |
|
||||
|-------------|-------|------|
|
||||
| Move proofs to appendix | 0.5-2 pages | Low — standard practice |
|
||||
| Condense related work | 0.5-1 page | Medium — may miss key citations |
|
||||
| Combine tables with subfigures | 0.25-0.5 page | Low — often improves readability |
|
||||
| Use `\vspace{-Xpt}` sparingly | 0.1-0.3 page | Low if subtle, high if obvious |
|
||||
| Remove qualitative examples | 0.5-1 page | Medium — reviewers like examples |
|
||||
| Reduce figure sizes | 0.25-0.5 page | High — figures must remain readable |
|
||||
|
||||
**Do NOT**: reduce font size, change margins, remove required sections (limitations, broader impact), or use `\small`/`\footnotesize` for main text.
|
||||
|
||||
### Step 5.10: Ethics & Broader Impact Statement
|
||||
|
||||
Most venues now require or strongly encourage an ethics/broader impact statement. This is not boilerplate — reviewers read it and can flag ethics concerns that trigger desk rejection.
|
||||
|
||||
**What to include:**
|
||||
|
||||
| Component | Content | Required By |
|
||||
|-----------|---------|-------------|
|
||||
| **Positive societal impact** | How your work benefits society | NeurIPS, ICML |
|
||||
| **Potential negative impact** | Misuse risks, dual-use concerns, failure modes | NeurIPS, ICML |
|
||||
| **Fairness & bias** | Does your method/data have known biases? | All venues (implicitly) |
|
||||
| **Environmental impact** | Compute carbon footprint for large-scale training | ICML, increasingly NeurIPS |
|
||||
| **Privacy** | Does your work use or enable processing of personal data? | ACL, NeurIPS |
|
||||
| **LLM disclosure** | Was AI used in writing or experiments? | ICLR (mandatory), ACL |
|
||||
|
||||
**Writing the statement:**
|
||||
|
||||
```latex
|
||||
\section*{Broader Impact Statement}
|
||||
% NeurIPS/ICML: after conclusion, does not count toward page limit
|
||||
|
||||
% 1. Positive applications (1-2 sentences)
|
||||
This work enables [specific application] which may benefit [specific group].
|
||||
|
||||
% 2. Risks and mitigations (1-3 sentences, be specific)
|
||||
[Method/model] could potentially be misused for [specific risk]. We mitigate
|
||||
this by [specific mitigation, e.g., releasing only model weights above size X,
|
||||
including safety filters, documenting failure modes].
|
||||
|
||||
% 3. Limitations of impact claims (1 sentence)
|
||||
Our evaluation is limited to [specific domain]; broader deployment would
|
||||
require [specific additional work].
|
||||
```
|
||||
|
||||
**Common mistakes:**
|
||||
- Writing "we foresee no negative impacts" (almost never true — reviewers distrust this)
|
||||
- Being vague: "this could be misused" without specifying how
|
||||
- Ignoring compute costs for large-scale work
|
||||
- Forgetting to disclose LLM use at venues that require it
|
||||
|
||||
**Compute carbon footprint** (for training-heavy papers):
|
||||
```python
|
||||
# Estimate using ML CO2 Impact tool methodology
|
||||
gpu_hours = 1000 # total GPU hours
|
||||
gpu_tdp_watts = 400 # e.g., A100 = 400W
|
||||
pue = 1.1 # Power Usage Effectiveness (data center overhead)
|
||||
carbon_intensity = 0.429 # kg CO2/kWh (US average; varies by region)
|
||||
|
||||
energy_kwh = (gpu_hours * gpu_tdp_watts * pue) / 1000
|
||||
carbon_kg = energy_kwh * carbon_intensity
|
||||
print(f"Energy: {energy_kwh:.0f} kWh, Carbon: {carbon_kg:.0f} kg CO2eq")
|
||||
```
|
||||
|
||||
### Step 5.11: Datasheets & Model Cards (If Applicable)
|
||||
|
||||
If your paper introduces a **new dataset** or **releases a model**, include structured documentation. Reviewers increasingly expect this, and NeurIPS Datasets & Benchmarks track requires it.
|
||||
|
||||
**Datasheets for Datasets** (Gebru et al., 2021) — include in appendix:
|
||||
|
||||
```
|
||||
Dataset Documentation (Appendix):
|
||||
- Motivation: Why was this dataset created? What task does it support?
|
||||
- Composition: What are the instances? How many? What data types?
|
||||
- Collection: How was data collected? What was the source?
|
||||
- Preprocessing: What cleaning/filtering was applied?
|
||||
- Distribution: How is the dataset distributed? Under what license?
|
||||
- Maintenance: Who maintains it? How to report issues?
|
||||
- Ethical considerations: Contains personal data? Consent obtained?
|
||||
Potential for harm? Known biases?
|
||||
```
|
||||
|
||||
**Model Cards** (Mitchell et al., 2019) — include in appendix for model releases:
|
||||
|
||||
```
|
||||
Model Card (Appendix):
|
||||
- Model details: Architecture, training data, training procedure
|
||||
- Intended use: Primary use cases, out-of-scope uses
|
||||
- Metrics: Evaluation metrics and results on benchmarks
|
||||
- Ethical considerations: Known biases, fairness evaluations
|
||||
- Limitations: Known failure modes, domains where model underperforms
|
||||
```
|
||||
|
||||
### Writing Style
|
||||
|
||||
**Sentence-level clarity (Gopen & Swan's 7 Principles):**
|
||||
|
||||
| Principle | Rule |
|
||||
|-----------|------|
|
||||
| Subject-verb proximity | Keep subject and verb close |
|
||||
| Stress position | Place emphasis at sentence ends |
|
||||
| Topic position | Put context first, new info after |
|
||||
| Old before new | Familiar info → unfamiliar info |
|
||||
| One unit, one function | Each paragraph makes one point |
|
||||
| Action in verb | Use verbs, not nominalizations |
|
||||
| Context before new | Set stage before presenting |
|
||||
|
||||
**Word choice (Lipton, Steinhardt):**
|
||||
- Be specific: "accuracy" not "performance"
|
||||
- Eliminate hedging: drop "may" unless genuinely uncertain
|
||||
- Consistent terminology throughout
|
||||
- Avoid incremental vocabulary: "develop", not "combine"
|
||||
|
||||
**Full writing guide with examples**: See [references/writing-guide.md](https://github.com/NousResearch/hermes-agent/blob/main/skills/research/research-paper-writing/references/writing-guide.md)
|
||||
|
||||
### Using LaTeX Templates
|
||||
|
||||
**Always copy the entire template directory first, then write within it.**
|
||||
|
||||
```
|
||||
Template Setup Checklist:
|
||||
- [ ] Step 1: Copy entire template directory to new project
|
||||
- [ ] Step 2: Verify template compiles as-is (before any changes)
|
||||
- [ ] Step 3: Read the template's example content to understand structure
|
||||
- [ ] Step 4: Replace example content section by section
|
||||
- [ ] Step 5: Use template macros (check preamble for \newcommand definitions)
|
||||
- [ ] Step 6: Clean up template artifacts only at the end
|
||||
```
|
||||
|
||||
**Step 1: Copy the Full Template**
|
||||
|
||||
```bash
|
||||
cp -r templates/neurips2025/ ~/papers/my-paper/
|
||||
cd ~/papers/my-paper/
|
||||
ls -la # Should see: main.tex, neurips.sty, Makefile, etc.
|
||||
```
|
||||
|
||||
Copy the ENTIRE directory, not just the .tex file. Templates include style files (.sty), bibliography styles (.bst), example content, and Makefiles.
|
||||
|
||||
**Step 2: Verify Template Compiles First**
|
||||
|
||||
Before making ANY changes:
|
||||
```bash
|
||||
latexmk -pdf main.tex
|
||||
# Or manual: pdflatex main.tex && bibtex main && pdflatex main.tex && pdflatex main.tex
|
||||
```
|
||||
|
||||
If the unmodified template doesn't compile, fix that first (usually missing TeX packages — install via `tlmgr install <package>`).
|
||||
|
||||
**Step 3: Keep Template Content as Reference**
|
||||
|
||||
Don't immediately delete example content. Comment it out and use as formatting reference:
|
||||
```latex
|
||||
% Template example (keep for reference):
|
||||
% \begin{figure}[t]
|
||||
% \centering
|
||||
% \includegraphics[width=0.8\linewidth]{example-image}
|
||||
% \caption{Template shows caption style}
|
||||
% \end{figure}
|
||||
|
||||
% Your actual figure:
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.8\linewidth]{your-figure.pdf}
|
||||
\caption{Your caption following the same style.}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
**Step 4: Replace Content Section by Section**
|
||||
|
||||
Work through systematically: title/authors → abstract → introduction → methods → experiments → related work → conclusion → references → appendix. Compile after each section.
|
||||
|
||||
**Step 5: Use Template Macros**
|
||||
|
||||
```latex
|
||||
\newcommand{\method}{YourMethodName} % Consistent method naming
|
||||
\newcommand{\eg}{e.g.,\xspace} % Proper abbreviations
|
||||
\newcommand{\ie}{i.e.,\xspace}
|
||||
```
|
||||
|
||||
### Template Pitfalls
|
||||
|
||||
| Pitfall | Problem | Solution |
|
||||
|---------|---------|----------|
|
||||
| Copying only `.tex` file | Missing `.sty`, won't compile | Copy entire directory |
|
||||
| Modifying `.sty` files | Breaks conference formatting | Never edit style files |
|
||||
| Adding random packages | Conflicts, breaks template | Only add if necessary |
|
||||
| Deleting template content early | Lose formatting reference | Keep as comments until done |
|
||||
| Not compiling frequently | Errors accumulate | Compile after each section |
|
||||
| Raster PNGs for figures | Blurry in paper | Always use vector PDF via `savefig('fig.pdf')` |
|
||||
|
||||
### Quick Template Reference
|
||||
|
||||
| Conference | Main File | Style File | Page Limit |
|
||||
|------------|-----------|------------|------------|
|
||||
| NeurIPS 2025 | `main.tex` | `neurips.sty` | 9 pages |
|
||||
| ICML 2026 | `example_paper.tex` | `icml2026.sty` | 8 pages |
|
||||
| ICLR 2026 | `iclr2026_conference.tex` | `iclr2026_conference.sty` | 9 pages |
|
||||
| ACL 2025 | `acl_latex.tex` | `acl.sty` | 8 pages (long) |
|
||||
| AAAI 2026 | `aaai2026-unified-template.tex` | `aaai2026.sty` | 7 pages |
|
||||
| COLM 2025 | `colm2025_conference.tex` | `colm2025_conference.sty` | 9 pages |
|
||||
|
||||
**Universal**: Double-blind, references don't count, appendices unlimited, LaTeX required.
|
||||
|
||||
Templates in `templates/` directory. See [templates/README.md](https://github.com/NousResearch/hermes-agent/blob/main/skills/research/research-paper-writing/templates/README.md) for compilation setup (VS Code, CLI, Overleaf, other IDEs).
|
||||
|
||||
### Tables and Figures
|
||||
|
||||
**Tables** — use `booktabs` for professional formatting:
|
||||
|
||||
```latex
|
||||
\usepackage{booktabs}
|
||||
\begin{tabular}{lcc}
|
||||
\toprule
|
||||
Method & Accuracy $\uparrow$ & Latency $\downarrow$ \\
|
||||
\midrule
|
||||
Baseline & 85.2 & 45ms \\
|
||||
\textbf{Ours} & \textbf{92.1} & 38ms \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Bold best value per metric
|
||||
- Include direction symbols ($\uparrow$ higher better, $\downarrow$ lower better)
|
||||
- Right-align numerical columns
|
||||
- Consistent decimal precision
|
||||
|
||||
**Figures**:
|
||||
- **Vector graphics** (PDF, EPS) for all plots and diagrams — `plt.savefig('fig.pdf')`
|
||||
- **Raster** (PNG 600 DPI) only for photographs
|
||||
- **Colorblind-safe palettes** (Okabe-Ito or Paul Tol)
|
||||
- Verify **grayscale readability** (8% of men have color vision deficiency)
|
||||
- **No title inside figure** — the caption serves this function
|
||||
- **Self-contained captions** — reader should understand without main text
|
||||
|
||||
### Conference Resubmission
|
||||
|
||||
For converting between venues, see Phase 7 (Submission Preparation) — it covers the full conversion workflow, page-change table, and post-rejection guidance.
|
||||
|
||||
### Professional LaTeX Preamble
|
||||
|
||||
Add these packages to any paper for professional quality. They are compatible with all major conference style files:
|
||||
|
||||
```latex
|
||||
% --- Professional Packages (add after conference style file) ---
|
||||
|
||||
% Typography
|
||||
\usepackage{microtype} % Microtypographic improvements (protrusion, expansion)
|
||||
% Makes text noticeably more polished — always include
|
||||
|
||||
% Tables
|
||||
\usepackage{booktabs} % Professional table rules (\toprule, \midrule, \bottomrule)
|
||||
\usepackage{siunitx} % Consistent number formatting, decimal alignment
|
||||
% Usage: \num{12345} → 12,345; \SI{3.5}{GHz} → 3.5 GHz
|
||||
% Table alignment: S column type for decimal-aligned numbers
|
||||
|
||||
% Figures
|
||||
\usepackage{graphicx} % Include graphics (\includegraphics)
|
||||
\usepackage{subcaption} % Subfigures with (a), (b), (c) labels
|
||||
% Usage: \begin{subfigure}{0.48\textwidth} ... \end{subfigure}
|
||||
|
||||
% Diagrams and Algorithms
|
||||
\usepackage{tikz} % Programmable vector diagrams
|
||||
\usetikzlibrary{arrows.meta, positioning, shapes.geometric, calc, fit, backgrounds}
|
||||
\usepackage[ruled,vlined]{algorithm2e} % Professional pseudocode
|
||||
% Alternative: \usepackage{algorithmicx} if template bundles it
|
||||
|
||||
% Cross-references
|
||||
\usepackage{cleveref} % Smart references: \cref{fig:x} → "Figure 1"
|
||||
% MUST be loaded AFTER hyperref
|
||||
% Handles: figures, tables, sections, equations, algorithms
|
||||
|
||||
% Math (usually included by conference .sty, but verify)
|
||||
\usepackage{amsmath,amssymb} % AMS math environments and symbols
|
||||
\usepackage{mathtools} % Extends amsmath (dcases, coloneqq, etc.)
|
||||
|
||||
% Colors (for figures and diagrams)
|
||||
\usepackage{xcolor} % Color management
|
||||
% Okabe-Ito colorblind-safe palette:
|
||||
\definecolor{okblue}{HTML}{0072B2}
|
||||
\definecolor{okorange}{HTML}{E69F00}
|
||||
\definecolor{okgreen}{HTML}{009E73}
|
||||
\definecolor{okred}{HTML}{D55E00}
|
||||
\definecolor{okpurple}{HTML}{CC79A7}
|
||||
\definecolor{okcyan}{HTML}{56B4E9}
|
||||
\definecolor{okyellow}{HTML}{F0E442}
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
- `microtype` is the single highest-impact package for visual quality. It adjusts character spacing at a sub-pixel level. Always include it.
|
||||
- `siunitx` handles decimal alignment in tables via the `S` column type — eliminates manual spacing.
|
||||
- `cleveref` must be loaded **after** `hyperref`. Most conference .sty files load hyperref, so put cleveref last.
|
||||
- Check if the conference template already loads any of these (especially `algorithm`, `amsmath`, `graphicx`). Don't double-load.
|
||||
|
||||
### siunitx Table Alignment
|
||||
|
||||
`siunitx` makes number-heavy tables significantly more readable:
|
||||
|
||||
```latex
|
||||
\begin{tabular}{l S[table-format=2.1] S[table-format=2.1] S[table-format=2.1]}
|
||||
\toprule
|
||||
Method & {Accuracy $\uparrow$} & {F1 $\uparrow$} & {Latency (ms) $\downarrow$} \\
|
||||
\midrule
|
||||
Baseline & 85.2 & 83.7 & 45.3 \\
|
||||
Ablation (no X) & 87.1 & 85.4 & 42.1 \\
|
||||
\textbf{Ours} & \textbf{92.1} & \textbf{90.8} & \textbf{38.7} \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
```
|
||||
|
||||
The `S` column type auto-aligns on the decimal point. Headers in `{}` escape the alignment.
|
||||
|
||||
### Subfigures
|
||||
|
||||
Standard pattern for side-by-side figures:
|
||||
|
||||
```latex
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\begin{subfigure}[b]{0.48\textwidth}
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{fig_results_a.pdf}
|
||||
\caption{Results on Dataset A.}
|
||||
\label{fig:results-a}
|
||||
\end{subfigure}
|
||||
\hfill
|
||||
\begin{subfigure}[b]{0.48\textwidth}
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{fig_results_b.pdf}
|
||||
\caption{Results on Dataset B.}
|
||||
\label{fig:results-b}
|
||||
\end{subfigure}
|
||||
\caption{Comparison of our method across two datasets. (a) shows the scaling
|
||||
behavior and (b) shows the ablation results. Both use 5 random seeds.}
|
||||
\label{fig:results}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
Use `\cref{fig:results}` → "Figure 1", `\cref{fig:results-a}` → "Figure 1a".
|
||||
|
||||
### Pseudocode with algorithm2e
|
||||
|
||||
```latex
|
||||
\begin{algorithm}[t]
|
||||
\caption{Iterative Refinement with Judge Panel}
|
||||
\label{alg:method}
|
||||
\KwIn{Task $T$, model $M$, judges $J_1 \ldots J_n$, convergence threshold $k$}
|
||||
\KwOut{Final output $A^*$}
|
||||
$A \gets M(T)$ \tcp*{Initial generation}
|
||||
$\text{streak} \gets 0$\;
|
||||
\While{$\text{streak} < k$}{
|
||||
$C \gets \text{Critic}(A, T)$ \tcp*{Identify weaknesses}
|
||||
$B \gets M(T, C)$ \tcp*{Revised version addressing critique}
|
||||
$AB \gets \text{Synthesize}(A, B)$ \tcp*{Merge best elements}
|
||||
\ForEach{judge $J_i$}{
|
||||
$\text{rank}_i \gets J_i(\text{shuffle}(A, B, AB))$ \tcp*{Blind ranking}
|
||||
}
|
||||
$\text{winner} \gets \text{BordaCount}(\text{ranks})$\;
|
||||
\eIf{$\text{winner} = A$}{
|
||||
$\text{streak} \gets \text{streak} + 1$\;
|
||||
}{
|
||||
$A \gets \text{winner}$; $\text{streak} \gets 0$\;
|
||||
}
|
||||
}
|
||||
\Return{$A$}\;
|
||||
\end{algorithm}
|
||||
```
|
||||
|
||||
### TikZ Diagram Patterns
|
||||
|
||||
TikZ is the standard for method diagrams in ML papers. Common patterns:
|
||||
|
||||
**Pipeline/Flow Diagram** (most common in ML papers):
|
||||
|
||||
```latex
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\begin{tikzpicture}[
|
||||
node distance=1.8cm,
|
||||
box/.style={rectangle, draw, rounded corners, minimum height=1cm,
|
||||
minimum width=2cm, align=center, font=\small},
|
||||
arrow/.style={-{Stealth[length=3mm]}, thick},
|
||||
]
|
||||
\node[box, fill=okcyan!20] (input) {Input\\$x$};
|
||||
\node[box, fill=okblue!20, right of=input] (encoder) {Encoder\\$f_\theta$};
|
||||
\node[box, fill=okgreen!20, right of=encoder] (latent) {Latent\\$z$};
|
||||
\node[box, fill=okorange!20, right of=latent] (decoder) {Decoder\\$g_\phi$};
|
||||
\node[box, fill=okred!20, right of=decoder] (output) {Output\\$\hat{x}$};
|
||||
|
||||
\draw[arrow] (input) -- (encoder);
|
||||
\draw[arrow] (encoder) -- (latent);
|
||||
\draw[arrow] (latent) -- (decoder);
|
||||
\draw[arrow] (decoder) -- (output);
|
||||
\end{tikzpicture}
|
||||
\caption{Architecture overview. The encoder maps input $x$ to latent
|
||||
representation $z$, which the decoder reconstructs.}
|
||||
\label{fig:architecture}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
**Comparison/Matrix Diagram** (for showing method variants):
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}[
|
||||
cell/.style={rectangle, draw, minimum width=2.5cm, minimum height=1cm,
|
||||
align=center, font=\small},
|
||||
header/.style={cell, fill=gray!20, font=\small\bfseries},
|
||||
]
|
||||
% Headers
|
||||
\node[header] at (0, 0) {Method};
|
||||
\node[header] at (3, 0) {Converges?};
|
||||
\node[header] at (6, 0) {Quality?};
|
||||
% Rows
|
||||
\node[cell] at (0, -1) {Single Pass};
|
||||
\node[cell, fill=okgreen!15] at (3, -1) {N/A};
|
||||
\node[cell, fill=okorange!15] at (6, -1) {Baseline};
|
||||
\node[cell] at (0, -2) {Critique+Revise};
|
||||
\node[cell, fill=okred!15] at (3, -2) {No};
|
||||
\node[cell, fill=okred!15] at (6, -2) {Degrades};
|
||||
\node[cell] at (0, -3) {Ours};
|
||||
\node[cell, fill=okgreen!15] at (3, -3) {Yes ($k$=2)};
|
||||
\node[cell, fill=okgreen!15] at (6, -3) {Improves};
|
||||
\end{tikzpicture}
|
||||
```
|
||||
|
||||
**Iterative Loop Diagram** (for methods with feedback):
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}[
|
||||
node distance=2cm,
|
||||
box/.style={rectangle, draw, rounded corners, minimum height=0.8cm,
|
||||
minimum width=1.8cm, align=center, font=\small},
|
||||
arrow/.style={-{Stealth[length=3mm]}, thick},
|
||||
label/.style={font=\scriptsize, midway, above},
|
||||
]
|
||||
\node[box, fill=okblue!20] (gen) {Generator};
|
||||
\node[box, fill=okred!20, right=2.5cm of gen] (critic) {Critic};
|
||||
\node[box, fill=okgreen!20, below=1.5cm of $(gen)!0.5!(critic)$] (judge) {Judge Panel};
|
||||
|
||||
\draw[arrow] (gen) -- node[label] {output $A$} (critic);
|
||||
\draw[arrow] (critic) -- node[label, right] {critique $C$} (judge);
|
||||
\draw[arrow] (judge) -| node[label, left, pos=0.3] {winner} (gen);
|
||||
\end{tikzpicture}
|
||||
```
|
||||
|
||||
### latexdiff for Revision Tracking
|
||||
|
||||
Essential for rebuttals — generates a marked-up PDF showing changes between versions:
|
||||
|
||||
```bash
|
||||
# Install
|
||||
# macOS: brew install latexdiff (or comes with TeX Live)
|
||||
# Linux: sudo apt install latexdiff
|
||||
|
||||
# Generate diff
|
||||
latexdiff paper_v1.tex paper_v2.tex > paper_diff.tex
|
||||
pdflatex paper_diff.tex
|
||||
|
||||
# For multi-file projects (with \input{} or \include{})
|
||||
latexdiff --flatten paper_v1.tex paper_v2.tex > paper_diff.tex
|
||||
```
|
||||
|
||||
This produces a PDF with deletions in red strikethrough and additions in blue — standard format for rebuttal supplements.
|
||||
|
||||
### SciencePlots for matplotlib
|
||||
|
||||
Install and use for publication-quality plots:
|
||||
|
||||
```bash
|
||||
pip install SciencePlots
|
||||
```
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import scienceplots # registers styles
|
||||
|
||||
# Use science style (IEEE-like, clean)
|
||||
with plt.style.context(['science', 'no-latex']):
|
||||
fig, ax = plt.subplots(figsize=(3.5, 2.5)) # Single-column width
|
||||
ax.plot(x, y, label='Ours', color='#0072B2')
|
||||
ax.plot(x, y2, label='Baseline', color='#D55E00', linestyle='--')
|
||||
ax.set_xlabel('Training Steps')
|
||||
ax.set_ylabel('Accuracy')
|
||||
ax.legend()
|
||||
fig.savefig('paper/fig_results.pdf', bbox_inches='tight')
|
||||
|
||||
# Available styles: 'science', 'ieee', 'nature', 'science+ieee'
|
||||
# Add 'no-latex' if LaTeX is not installed on the machine generating plots
|
||||
```
|
||||
|
||||
**Standard figure sizes** (two-column format):
|
||||
- Single column: `figsize=(3.5, 2.5)` — fits in one column
|
||||
- Double column: `figsize=(7.0, 3.0)` — spans both columns
|
||||
- Square: `figsize=(3.5, 3.5)` — for heatmaps, confusion matrices
|
||||
|
||||
---
|
||||
The complete drafting procedure (section-by-section order, LaTeX scaffolding, figure/table
|
||||
conventions, abstract and intro formulas, related-work positioning) lives in
|
||||
`references/phase5-paper-drafting.md` — load it with `read_file` when you reach this phase.
|
||||
Pair it with `references/writing-guide.md` for prose-level style rules.
|
||||
|
||||
## Phase 6: Self-Review & Revision
|
||||
|
||||
|
|
|
|||
|
|
@ -14,8 +14,8 @@ Run 150+ AI apps (image, video, LLM) via inference.sh CLI.
|
|||
|
||||
| | |
|
||||
|---|---|
|
||||
| Source | Optional — install with `hermes skills install official/devops/cli` |
|
||||
| Path | `optional-skills/devops/cli` |
|
||||
| Source | Optional — install with `hermes skills install official/devops/inference-sh-cli` |
|
||||
| Path | `optional-skills/devops/inference-sh-cli` |
|
||||
| Version | `1.0.0` |
|
||||
| Author | okaris |
|
||||
| License | MIT |
|
||||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Huggingface Accelerate — Run PyTorch training across GPUs with minimal changes"
|
||||
sidebar_label: "Huggingface Accelerate"
|
||||
title: "Accelerate — Run PyTorch training across GPUs with minimal changes"
|
||||
sidebar_label: "Accelerate"
|
||||
description: "Run PyTorch training across GPUs with minimal changes"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Huggingface Accelerate
|
||||
# Accelerate
|
||||
|
||||
Run PyTorch training across GPUs with minimal changes.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Optimizing Attention Flash — Speed up long-sequence transformer training and inference"
|
||||
sidebar_label: "Optimizing Attention Flash"
|
||||
title: "Flash Attention — Speed up long-sequence transformer training and inference"
|
||||
sidebar_label: "Flash Attention"
|
||||
description: "Speed up long-sequence transformer training and inference"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Optimizing Attention Flash
|
||||
# Flash Attention
|
||||
|
||||
Speed up long-sequence transformer training and inference.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Lambda Labs Gpu Cloud — On-demand GPU cloud instances for ML training"
|
||||
sidebar_label: "Lambda Labs Gpu Cloud"
|
||||
title: "Lambda Labs — On-demand GPU cloud instances for ML training"
|
||||
sidebar_label: "Lambda Labs"
|
||||
description: "On-demand GPU cloud instances for ML training"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Lambda Labs Gpu Cloud
|
||||
# Lambda Labs
|
||||
|
||||
On-demand GPU cloud instances for ML training.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Modal Serverless Gpu — Serverless GPU cloud for ML jobs and model APIs"
|
||||
sidebar_label: "Modal Serverless Gpu"
|
||||
title: "Modal — Serverless GPU cloud for ML jobs and model APIs"
|
||||
sidebar_label: "Modal"
|
||||
description: "Serverless GPU cloud for ML jobs and model APIs"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Modal Serverless Gpu
|
||||
# Modal
|
||||
|
||||
Serverless GPU cloud for ML jobs and model APIs.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Peft Fine Tuning — Fine-tune large LLMs with LoRA on limited GPU memory"
|
||||
sidebar_label: "Peft Fine Tuning"
|
||||
title: "Peft — Fine-tune large LLMs with LoRA on limited GPU memory"
|
||||
sidebar_label: "Peft"
|
||||
description: "Fine-tune large LLMs with LoRA on limited GPU memory"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Peft Fine Tuning
|
||||
# Peft
|
||||
|
||||
Fine-tune large LLMs with LoRA on limited GPU memory.
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Qdrant Vector Search — Vector search engine for production RAG systems"
|
||||
sidebar_label: "Qdrant Vector Search"
|
||||
title: "Qdrant — Vector search engine for production RAG systems"
|
||||
sidebar_label: "Qdrant"
|
||||
description: "Vector search engine for production RAG systems"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Qdrant Vector Search
|
||||
# Qdrant
|
||||
|
||||
Vector search engine for production RAG systems.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Sparse Autoencoder Training — Train sparse autoencoders to interpret model features"
|
||||
sidebar_label: "Sparse Autoencoder Training"
|
||||
title: "Saelens — Train sparse autoencoders to interpret model features"
|
||||
sidebar_label: "Saelens"
|
||||
description: "Train sparse autoencoders to interpret model features"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Sparse Autoencoder Training
|
||||
# Saelens
|
||||
|
||||
Train sparse autoencoders to interpret model features.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Simpo Training — Reference-free preference alignment, simpler than DPO"
|
||||
sidebar_label: "Simpo Training"
|
||||
title: "Simpo — Reference-free preference alignment, simpler than DPO"
|
||||
sidebar_label: "Simpo"
|
||||
description: "Reference-free preference alignment, simpler than DPO"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Simpo Training
|
||||
# Simpo
|
||||
|
||||
Reference-free preference alignment, simpler than DPO.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Slime Rl Training — RL post-training for LLMs with Megatron and SGLang"
|
||||
sidebar_label: "Slime Rl Training"
|
||||
title: "Slime — RL post-training for LLMs with Megatron and SGLang"
|
||||
sidebar_label: "Slime"
|
||||
description: "RL post-training for LLMs with Megatron and SGLang"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Slime Rl Training
|
||||
# Slime
|
||||
|
||||
RL post-training for LLMs with Megatron and SGLang.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Stable Diffusion Image Generation — Text-to-image generation, inpainting, and img2img"
|
||||
sidebar_label: "Stable Diffusion Image Generation"
|
||||
title: "Stable Diffusion — Text-to-image generation, inpainting, and img2img"
|
||||
sidebar_label: "Stable Diffusion"
|
||||
description: "Text-to-image generation, inpainting, and img2img"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Stable Diffusion Image Generation
|
||||
# Stable Diffusion
|
||||
|
||||
Text-to-image generation, inpainting, and img2img.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Distributed Llm Pretraining Torchtitan — Pretrain LLMs at scale with PyTorch 4D parallelism"
|
||||
sidebar_label: "Distributed Llm Pretraining Torchtitan"
|
||||
title: "Torchtitan — Pretrain LLMs at scale with PyTorch 4D parallelism"
|
||||
sidebar_label: "Torchtitan"
|
||||
description: "Pretrain LLMs at scale with PyTorch 4D parallelism"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Distributed Llm Pretraining Torchtitan
|
||||
# Torchtitan
|
||||
|
||||
Pretrain LLMs at scale with PyTorch 4D parallelism.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Fine Tuning With Trl — TRL: SFT, DPO, GRPO, RLOO reward modeling for LLM RLHF"
|
||||
sidebar_label: "Fine Tuning With Trl"
|
||||
title: "Trl Fine Tuning — TRL: SFT, DPO, GRPO, RLOO reward modeling for LLM RLHF"
|
||||
sidebar_label: "Trl Fine Tuning"
|
||||
description: "TRL: SFT, DPO, GRPO, RLOO reward modeling for LLM RLHF"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Fine Tuning With Trl
|
||||
# Trl Fine Tuning
|
||||
|
||||
TRL: SFT, DPO, GRPO, RLOO reward modeling for LLM RLHF.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Here.Now — Publish sites to {slug}.here.now and store files in Drives"
|
||||
sidebar_label: "Here.Now"
|
||||
title: "Here Now — Publish sites to {slug}.here.now and store files in Drives"
|
||||
sidebar_label: "Here Now"
|
||||
description: "Publish sites to {slug}.here.now and store files in Drives"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Here.Now
|
||||
# Here Now
|
||||
|
||||
Publish sites to {slug}.here.now and store files in Drives.
|
||||
|
||||
|
|
|
|||
|
|
@ -14,8 +14,8 @@ description: "通过 inference 运行 150+ AI 应用"
|
|||
|
||||
| | |
|
||||
|---|---|
|
||||
| 来源 | 可选——使用 `hermes skills install official/devops/cli` 安装 |
|
||||
| 路径 | `optional-skills/devops/cli` |
|
||||
| 来源 | 可选——使用 `hermes skills install official/devops/inference-sh-cli` 安装 |
|
||||
| 路径 | `optional-skills/devops/inference-sh-cli` |
|
||||
| 版本 | `1.0.0` |
|
||||
| 作者 | okaris |
|
||||
| 许可证 | MIT |
|
||||
|
|
@ -405,9 +405,9 @@ const sidebars: SidebarsConfig = {
|
|||
collapsed: true,
|
||||
items: [
|
||||
'user-guide/skills/optional/devops/devops-actual-setup',
|
||||
'user-guide/skills/optional/devops/devops-cli',
|
||||
'user-guide/skills/optional/devops/devops-docker-management',
|
||||
'user-guide/skills/optional/devops/devops-hermes-s6-container-supervision',
|
||||
'user-guide/skills/optional/devops/devops-inference-sh-cli',
|
||||
'user-guide/skills/optional/devops/devops-pinggy-tunnel',
|
||||
'user-guide/skills/optional/devops/devops-watchers',
|
||||
],
|
||||
|
|
|
|||
Loading…
Reference in New Issue