研究 · 实地笔记
Pulpie：用于清理网页的帕累托最优模型
作者
Bhavnick Minhas、Shreyash Nigam
所属机构
Feyn Labs
发布时间
2026年6月25日
阅读时长
13分钟
我们推出 Pulpie，这是一系列用于从 HTML 页面中提取主要内容的帕累托最优模型。Pulpie 以二十分之一的成本达到了 SOTA 级别的提取质量。
我们最小的模型 pulpie-orange-small 在 WebMainBench 上取得了 0.862 的 ROUGE-5 F1 分数。这与领先的提取器 Dripper（0.864 分）相当。Pulpie 取得这一成绩，尽管其规模仅为 Dripper 的三分之一：2.1 亿参数对比 Dripper 的 6 亿参数。
性能提升源于架构设计。Pulpie 是一个编码器，能在单次前向传播中将每个 HTML 块标记为内容或样板代码。这也使其速度很快。
在 NVIDIA L4 GPU 上，pulpie-orange-small 每秒可处理 13.7 个页面，而 Dripper 仅为每秒 0.68 个页面。以 L4 实例每小时 0.39 美元计算，清理 10 亿页面的成本，Pulpie 为 7,900 美元，Dripper 则为 159,000 美元。
Pulpie 实现了以往不可能达到的大规模高质量网页提取。我们预计这将惠及预训练和上下文管理。
我们的模型是开源的，可在 Hugging Face 上获取。使用说明请参见相关文档。
提取是瓶颈
语言模型会两次消耗网页内容。第一次是在预训练阶段，它们借此了解世界。第二次是在推理阶段，它们会拉取相关上下文。这两次输入大多都是噪声。在探索过程中，我们发现典型 HTML 页面上 70% 的块都是导航、广告、侧边栏和页脚等样板内容。主要内容仅占页面的一小部分。
然而，这一小部分内容却决定了模型在两端（预训练和推理）的质量。
AICC（Ma 等人，2025 年）衡量了更干净的提取对预训练的影响。该团队从同一个 Common Crawl 快照中构建了两个语料库。一个使用启发式方法提取内容。另一个使用基于模型的解析器提取内容。数据流水线中的其他所有环节保持一致。然后，他们在每个语料库上训练了相同的模型。
在基于模型提取的语料库上训练的模型，在 13 个基准测试中的平均准确率高出 1.08 个百分点。由于只有提取逻辑发生了变化，我们可以将这一提升完全归因于更干净的数据。
令人印象深刻的是，同一模型还击败了在 FineWeb 和 RefinedWeb 上训练的模型，这两个是经过最严格过滤的预训练语料库之一。这些数据集通过精心的过滤和去重赢得了声誉。通过改进提取器来超越它们，充分体现了干净数据的高价值。
除了设定一个低基准之外，糟糕的提取还会实质性地损害模型。启发式方法会破坏结构化内容。下表展示了 Trafilatura 和基于模型的提取器在保留代码块和公式方面的对比。低相似度分数表明存在损坏。如果在训练中使用，生成的模型将继承这种损伤。
| 内容 | Trafilatura（启发式） | 基于模型 |
|---|---|---|
| 代码块 | 0.13 | 0.91 |
| 公式 | 0.61 | 0.94 |
数据质量在推理阶段也同样重要。Shi 等人（ICML 2023）表明，仅一条不相关的段落就足以扰乱模型的回答。当模型的上下文没有噪声时，它的准确性和效率都会更高。
低成本清理
清理网络数据在训练和推理中都能带来回报。悬而未决的问题是，我们如何在大规模场景下做好清理工作？
首先，为了理解整体情况，我们可以根据一个问题将当前的提取器分为两类：该方法是在读取页面内容，还是在检查其结构？
基于结构的提取器通过表面信号来判断 HTML 块。它们对标签、DOM 和文本密度应用规则，以将内容与样板代码分离。Trafilatura、Readability 和 magic-html 就是采用这种方式。Boilerpipe 更进一步，在这些相同信号上训练了一个分类器。这些提取器易于运行，但它们会混淆结构相似的元素。一个导航表格和一个数据表格，在计算单元格数量的算法看来是完全一样的。
读取型提取器将页面输入给 Transformer 架构，并根据每个块的内容为其打上标签。Dripper 是一个基于此思想构建的解码器。该解码器一次生成一个 token 的标签。每个标签都迫使完整模型从内存中被读取一次，以完成单步工作。这使得速度受限于内存带宽，并导致运行成本高昂。
Pulpie 保留了阅读式方法，但将瓶颈转移到了计算层面。我们通过使用一种编码器架构来实现这一点，该架构能在单次前向传播中为每个块打上标签。这使得 Pulpie 能够在更小、更快、更便宜的同时，达到与 Dripper 相当的质量。
网页内容提取的质量与成本
剥离原始 HTML
完整流程分四个阶段运行：
- 简化 HTML。移除脚本、样式表及其他格式噪声。为每个块标记唯一 ID。
- 对块进行分块。将块拆分、进行 token 化，并打包成最多 8192 个 token 的块，以便每个块都能在单次模型处理中适配。约 80% 的页面可放入单个块中。
- 分类。运行一次前向传播。Pulpie 将每个块标记为内容或样板文件。
- 返回。将保留的块以 HTML 形式返回，或将其转换为 Markdown。
训练
训练 Pulpie 需要一个包含大量 HTML 页面及其块级标签的数据集。由于不存在这样的公开数据集，我们自行构建了一个。
我们从 Common Crawl 中采样了 16,670 个英文页面，并限制每个域名只取一个。然后，我们使用 MinerU-HTML 将每个页面拆分为块，并使用 DeepSeek V3.2 将每个块标记为内容或样板文件。进一步过滤移除了空页面、损坏页面及其他不合格页面，最终剩下 15,880 个页面。
随后，我们使用 Dripper 0.6B 作为第二个标注器，对所有 15,880 个页面进行标注，以标记不一致的标签。与 DeepSeek 的块级一致率为 93.3%。我们保留了两位标注器在至少 70% 的块上达成一致的 14,959 个页面，以此用部分数据换取更干净的训练集。
训练教师模型
为了创建我们的教师模型，我们在前述 14,959 个页面上对 EuroBERT-2.1B 进行了微调。
| 设置 | 数值 |
|---|---|
| 学习率 | 2e-5 |
| 有效批量大小 | 8 |
| 损失函数 | 类别加权交叉熵 |
| 硬件 | 4 块 A100 |
类别权重根据 28.6% 的内容比例进行反向设置，以应对不平衡问题。
该教师模型在 WebMainBench 英文测试集上取得了 0.873 的 ROUGE-5 F1 分数。拥有 21 亿参数，它虽然准确但运行成本高昂，因此我们将其蒸馏为更小的模型。
知识传授
为了更适配生产环境，我们将这个 21 亿参数的教师模型蒸馏为两个更小的模型：
- Pulpie Orange Base，一个 610M 参数的编码器。
- Pulpie Orange Small，一个 210M 参数的编码器。
两个学生模型都按照 Hinton 等人（2015）的方法向教师模型学习。教师模型经过软化的输出分布通过 KL 散度损失（权重为 0.7）提供了大部分信号，硬标签交叉熵损失（权重为 0.3）作为补充，温度参数设为 2.0。两者都在与教师模型相同的数据上进行训练。
蒸馏后的模型几乎保留了教师模型的全部质量。
| 模型 | 参数 | ROUGE-5 F1 | 与教师模型对比 |
|---|---|---|---|
| Pulpie Orange Small | 210M | 0.862 | -1.1 F1 点 |
| Dripper | 0.6B | 0.864 | -0.9 F1 点 |
| Pulpie Orange Base | 610M | 0.863 | -1.0 F1 点 |
| Pulpie Orange Large（教师模型） | 2.1B | 0.873 | - |
尽管规模缩小了十倍，210M 参数的模型与教师模型的差距仍在 1 个 F1 点以内。结合其速度和成本优势，pulpie-orange-small 在整个系列中拥有最佳的规模与质量比。这是我们推荐用于生产环境的模型。
结果
质量
我们在 WebMainBench 的英文子集（涵盖所有难度等级的 6,647 个页面）上测量 ROUGE-5 F1 分数。空提取结果计为零分。
| 方法 | ROUGE-5 F1 | 空页面数 |
|---|---|---|
| magic-html | 0.700 | 384 |
| Trafilatura | 0.619 | 16 |
| Pulpie Orange Small | 0.862 | 45 |
| Dripper | 0.864 | 135 |
| Pulpie Orange Base | 0.863 | 36 |
| Pulpie Orange Large | 0.873 | 21 |
Pulpie Orange Large 是最强的单模型，得分为 0.873，领先 Dripper 0.9 个 F1 点。210M 参数的模型在规模仅为 Dripper 三分之一的情况下与之持平。前沿大语言模型在此基准测试上得分更高，接近 0.90，这也是 Pulpie 正在接近的质量水平。
Dripper 在 135 个页面上返回了空结果。其中 130 个页面是因为页面内容超出了其 32k token 的上下文窗口。Pulpie 将内容块打包成 8,192 token 的片段，因此页面长度不会导致处理失败。
按难度细分结果：
| 方法 | 全部 | 简单 | 中等 | 困难 |
|---|---|---|---|---|
| magic-html | 0.700 | 0.773 | 0.697 | 0.637 |
| Trafilatura | 0.619 | 0.721 | 0.619 | 0.526 |
| Pulpie Orange Small | 0.862 | 0.906 | 0.868 | 0.813 |
| Dripper | 0.864 | 0.913 | 0.865 | 0.817 |
| Pulpie Orange Base | 0.863 | 0.906 | 0.868 | 0.818 |
| Pulpie Orange Large | 0.873 | 0.914 | 0.879 | 0.827 |
随着页面难度增加，每种方法的表现都会下降。基于启发式规则的方法下降最快，从简单到困难页面 F1 分数下降了 14 到 20 个点，而编码器模型则下降了约 9 个 F1 点。Dripper 的性能范围与编码器模型相当，在简单和困难页面之间存在 10 个 F1 点的差距。
速度
各模型吞吐量
在 L4 上比 Dripper 快 20 倍，对比的是相同页面上的 Pulpie Small。
L4 吞吐量，基于 500 个真实 Common Crawl 页面：
| 方法 | 吞吐量（页面/秒） | 硬件 |
|---|---|---|
| Pulpie Orange Small | 13.7 | L4 |
| Dripper | 0.68 | L4 |
| Pulpie Orange Base | 3.9 | L4 |
| Pulpie Orange Large | 1.3 | L4 |
Pulpie Orange Small 在相同 L4 上运行速度比 Dripper 快 20 倍。
A100 吞吐量，相同页面，仅 GPU 推理，每个模型均采用批处理：
| 方法 | 吞吐量（页面/秒） | 硬件 |
|---|---|---|
| Pulpie Orange Small | 25.7 | A100 |
| Dripper | 3.6 | A100 |
| Pulpie Orange Base | 7.7 | A100 |
| Pulpie Orange Large | 3.5 | A100 |
在 A100 上，Pulpie Orange Small 运行速度比 Dripper 快 7.1 倍。2.1B 的教师模型在速度上与 Dripper 持平，同时在质量上超越它。
成本
每 10 亿页成本
在 L4 上比 Dripper 便宜 20 倍，对比的是相同页面上的 Pulpie Small。
10 亿页的 L4 成本，按 $0.39/小时计算。根据上述测量的吞吐量计算：
| 配置 | 页面/秒 | GPU 小时数 / 10 亿 | 成本 / 10 亿页 |
|---|---|---|---|
| L4 上的 Pulpie Small | 13.7 | 20,300 | ~$7,900 |
| L4 上的 Dripper | 0.68 | 408,000 | ~$159,000 |
| L4 上的 Pulpie Base | 3.9 | 71,200 | ~$28,000 |
| L4 上的 Pulpie Large | 1.3 | 214,000 | ~$83,000 |
10 亿页的 A100 成本，按 $2.72/小时计算。根据上述测量的吞吐量计算：
| 配置 | 页面/秒 | GPU 小时数 / 10 亿 | 成本 / 10 亿页 |
|---|---|---|---|
| A100 上的 Pulpie Small | 25.7 | 10,800 | ~$29,000 |
| A100 上的 Dripper | 3.6 | 77,200 | ~$210,000 |
| A100 上的 Pulpie Base | 7.7 | 36,100 | ~$98,000 |
| A100 上的 Pulpie Large | 3.5 | 79,400 | ~$216,000 |
像编码器一样廉价的 GPU
Pulpie 和 Dripper 之间的吞吐量差距远大于 3 倍的模型大小差异所暗示的。在 A100 上，我们测得这一差距为 7.1 倍，而在 L4 上则扩大到 20 倍。其原因在于架构。
解码器每次生成一个 token 的标签。每一步都需要从 GPU 内存中读取整个模型以生成单个 token。因此，解码器的速度受限于内存带宽。相反，编码器对整个输入执行一次前向传播。这种密集的矩阵乘法仅受限于算力。
此外，A100 和 L4 在带宽上的差异大于在算力上的差异：
| 维度 | NVIDIA A100 | NVIDIA L4 | 比率 (A100/L4) |
|---|---|---|---|
| 内存带宽 | 2,039 GB/s | 300 GB/s | ~6.8x |
| Tensor Core TFLOPS | 312 | 120 | ~2.6x |
从 A100 降级到 L4 对带宽受限的解码器的限制远大于对计算受限的编码器。这拉大了吞吐量差距，使得 Pulpie Orange Large 在 L4 上能够领先，尽管在 A100 上与 Dripper 性能相当。
Pulpie 模型已上线 Hugging Face。安装该包：
pip install pulpie从原始 HTML 中提取干净内容：
from pulpie import Extractor
extractor = Extractor() # defaults to Pulpie Orange Small
result = extractor.extract(html)
print(result.markdown) # clean markdown
print(result.n_main, result.n_other) # blocks kept vs dropped若追求最高质量而非速度，请选择更大的模型：
extractor = Extractor(model="large") # "small" (default), "base", or "large"对于批量处理，该流程可在单个或多个 GPU 上实现 CPU 预处理与 GPU 推理的重叠：
from pulpie import Pipeline, PageInput
pipeline = Pipeline(model="small")
results = pipeline.extract_batch(
[PageInput(html=h, page_id=i) for i, h in enumerate(pages)]
)所有三个模型均基于 EuroBERT（Boizard 等人，2025），使用相同的 <|sep|> 块标记架构，并共享同一个 tokenizer：
| 名称 | Hugging Face | 参数量 | ROUGE-5 F1 | 备注 |
|---|---|---|---|---|
| Orange Small | feyninc/pulpie-orange-small-v1 | 2.1 亿 | 0.862 | 推荐 |
| Orange Base | feyninc/pulpie-orange-base-v1 | 6.1 亿 | 0.863 | 从 Large 蒸馏而来 |
| Orange Large | feyninc/pulpie-orange-large-v1 | 21 亿 | 0.873 | 教师模型 |
Pulpie Orange Small 是推荐且默认的模型。它以二十分之一的成本接近 SOTA 提取质量，并且运行速度最快。
Pulpie 由 Feyn 构建。您可以在 GitHub、Hugging Face 或 X 上找到我们。
致谢
Pulpie 直接基于 MinerU-HTML 和 Dripper 团队（Ma 等人，2025）的工作构建。他们的 simplify_html 预处理、块级标注方案以及 WebMainBench 基准测试是此项工作的基础。我们还使用他们的 Dripper 0.6B 模型来交叉验证我们的训练标签。我们感谢他们公开了自己的工具和数据。
参考文献
[1]
Ma 等人。"AICC: Parse HTML Finer, Make Models Better — A 7.3T AI-Ready Corpus Built by a Model-Based HTML Parser."
arXiv:2511.16397
(2025).
[2]
Boizard 等人。"EuroBERT: Scaling Multilingual Encoders for European Languages."
arXiv:2503.05500
(2025).
[3]
Hinton 等人。"Distilling the Knowledge in a Neural Network."
arXiv:1503.02531
(2015).
[4]
Raffel 等人。"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer."
JMLR
2020.
[5]
Penedo 等人。"The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only."
NeurIPS
2023.
[6]
Penedo 等人。"The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale."
arXiv:2406.17557
(2024).
[7]
Li 等人。"DataComp-LM: In Search of the Next Generation of Training Sets for Language Models."
NeurIPS
2024.
[8]
Soldaini 等人著《Dolma：一个用于语言模型预训练研究的三万亿 token 开放语料库》。
ACL
2024 年。
[9]
Barbaresi 著《Trafilatura：用于文本发现与提取的网络爬取库及命令行工具》。
ACL/IJCNLP
2021 年。
[10]
Kohlschütter 等人著《基于浅层文本特征的模板检测》。
WSDM
2010 年。
[11]
Pomikálek 著《从网络语料库中去除模板与重复内容》。
博士论文，马萨里克大学
，2011 年。
[12]
Bevendorff 等人著《网络内容提取算法的实证比较》。
SIGIR
2023 年。
[13]
Shi 等人著《大语言模型易受无关上下文干扰》。
ICML
2023 年。
引用此注释
@note{pulpie2026,
title = {Pulpie: Pareto-Optimal Models for Cleaning the Web},
author = {Minhas, Bhavnick and Nigam, Shreyash and Feyn Research},
year = {2026},
venue = {Feyn Field Notes}
} Research · field note
Pulpie: Pareto-Optimal Models for Cleaning the Web
Authors
Bhavnick Minhas, Shreyash Nigam
Affiliation
Feyn Labs
Published
June 25, 2026
Reading time
13 min
We’re introducing Pulpie, a family of Pareto-optimal models for extracting main content from HTML pages. Pulpie approaches SOTA extraction quality at one twentieth the cost.
Our smallest model, pulpie-orange-small, scores 0.862 ROUGE-5 F1 on WebMainBench. This matches Dripper, the leading extractor, which scores 0.864. Pulpie’s performance is despite it being a third the size: 210M parameters versus Dripper’s 600M.
The gains come from architecture. Pulpie is an encoder that labels every HTML block as content or boilerplate in a single forward pass. This also makes it fast.
On an NVIDIA L4 GPU, pulpie-orange-small processes 13.7 pages/sec against Dripper’s 0.68 pages/sec. At $0.39/hr for an L4 instance, cleaning 1 billion pages costs $7,900 with Pulpie and $159,000 with Dripper.
Pulpie unlocks high quality web extraction at a scale impossible before. We expect this to benefit pre-training and context management.
Our models are open source and available on Hugging Face. See for instructions.
Extraction is the bottleneck
Language models consume the web twice. First in pre-training, where they learn about the world. Then at inference, when they pull in relevant context. Both times the input is mostly noise. During discovery, we found 70% of the blocks on a typical HTML page hold boilerplate like navigation, ads, sidebars, and footers. Main content is only a small fraction of the page.
However, that fraction determines model quality on both ends.
AICC (Ma et al., 2025) measured the effect of cleaner extraction on pre-training. The team built two corpora from the same Common Crawl snapshot. One extracted content with heuristics. The other extracted it with a model-based parser. Everything else in the data pipeline remained equal. They then trained an identical model on each corpus.
The model trained on the model-extracted corpus scored 1.08 percentage points higher in average accuracy across 13 benchmarks. Since only extraction logic changed, we can attribute the gain entirely to having cleaner data.
Impressively, the same model also beat models trained on FineWeb and RefinedWeb, two of the most heavily filtered pre-training corpora. These datasets have earned their reputations through elaborate filtering and deduplication. Beating them by improving the extractor illustrates the high value of clean data.
Beyond setting a low baseline, poor extraction materially harms models. Heuristics break structured content. The table below shows how Trafilatura and model-based extractors compare on preserving code blocks and formulas. Low similarity scores indicate corruption. If used in training, resulting models will inherit this damage.
| Content | Trafilatura (heuristic) | Model-based |
|---|---|---|
| Code blocks | 0.13 | 0.91 |
| Formulas | 0.61 | 0.94 |
Data quality matters at inference too. Shi et al. (ICML 2023) showed that a single irrelevant passage is enough to derail a model’s answer. A model is more accurate and more efficient when its context is free of noise.
Cleaning on a budget
Cleaning the web pays off in both training and inference. The open question is how do we clean well at scale?
First, to understand the landscape, we can divide current extractors into two families based on the question: Does the method read the page, or inspect its structure?
Structure-based extractors judge an HTML block by surface signals. They apply rules over tags, DOM, and text density to separate content from boilerplate. Trafilatura, Readability, and magic-html work this way. Boilerpipe goes one step further and trains a classifier on those same signals. These extractors are easy to run but they confuse similarly built elements. A navigation table and a data table look identical to an algorithm counting cells.
Reading extractors feed the page to a transformer and label each block based on its content. Dripper is a decoder built on this idea. The decoder emits labels one token at a time. Each label forces the full model to be read from memory for a single step of work. This ties speed to memory bandwidth and makes runs expensive.
Pulpie keeps the reading approach but moves the bottleneck to compute. We do this by using an encoder architecture that labels every block in a single forward pass. This enables Pulpie to match Dripper’s quality while being smaller, faster, and cheaper.
Quality vs Cost of Web Content Extraction
Depulping raw HTML
The full pipeline runs in four stages:
- Simplify the HTML. Remove scripts, styles, and other formatting noise. Tag each block with a unique ID.
- Chunk the blocks. Split the blocks, tokenize them, and pack them into chunks of at most 8,192 tokens, so each chunk fits the model in one pass. About 80% of pages fit in a single chunk.
- Classify. Run a forward pass. Pulpie labels each block as content or boilerplate.
- Return. Return the kept blocks as HTML, or convert them to Markdown.
Training
Training Pulpie needed a large set of HTML pages with block-level labels. No such public set existed, so we built one.
We sampled 16,670 English pages from Common Crawl, limiting to one per domain. We then used MinerU-HTML to split each page into blocks, and labeled each block as content or boilerplate with DeepSeek V3.2. Further filtering removed empty, corrupted, and otherwise unfit pages, leaving 15,880.
We then ran Dripper 0.6B as a second labeler across all 15,880 pages to flag inconsistent labels. Block-level agreement with DeepSeek was 93.3%. We kept the 14,959 pages where the two labelers agreed on at least 70% of blocks, trading some data for a cleaner training set.
Teaching a teacher
To create our teacher model, we fine-tuned EuroBERT-2.1B on the aforementioned 14,959 pages.
| Setting | Value |
|---|---|
| Learning rate | 2e-5 |
| Effective batch size | 8 |
| Loss | Class-weighted cross-entropy |
| Hardware | 4x A100 |
Class weights are set inversely to the 28.6% content rate to counter the imbalance.
The teacher scored 0.873 ROUGE-5 F1 on the WebMainBench English set. At 2.1B parameters it is accurate but expensive to run, so we distilled it into smaller models.
Imparting knowledge
For a better production fit, we distilled the 2.1B teacher into two smaller models:
- Pulpie Orange Base, a 610M parameter encoder.
- Pulpie Orange Small, a 210M parameter encoder.
Both students learn from the teacher following Hinton et al. (2015). The teacher’s softened output distribution supplies most of the signal through a KL-divergence loss weighted 0.7, with hard-label cross-entropy making up the remaining 0.3, at temperature 2.0. Both train on the same data as the teacher.
The distilled models keep almost all of the teacher’s quality.
| Model | Parameters | ROUGE-5 F1 | vs. Teacher |
|---|---|---|---|
| Pulpie Orange Small | 210M | 0.862 | -1.1 F1 points |
| Dripper | 0.6B | 0.864 | -0.9 F1 points |
| Pulpie Orange Base | 610M | 0.863 | -1.0 F1 points |
| Pulpie Orange Large (teacher) | 2.1B | 0.873 | - |
Despite a tenfold cut in size, the 210M model is within one F1 point. Combined with its speed and cost benefits, pulpie-orange-small features the best size-to-quality ratio in the entire family. It is the model we recommend for production use.
Results
Quality
We measure ROUGE-5 F1 on the English subset of WebMainBench (6,647 pages across all difficulty levels). Empty extractions count as zero.
| Method | ROUGE-5 F1 | Empty pages |
|---|---|---|
| magic-html | 0.700 | 384 |
| Trafilatura | 0.619 | 16 |
| Pulpie Orange Small | 0.862 | 45 |
| Dripper | 0.864 | 135 |
| Pulpie Orange Base | 0.863 | 36 |
| Pulpie Orange Large | 0.873 | 21 |
Pulpie Orange Large is the strongest single model at 0.873, ahead of Dripper by 0.9 F1 points. The 210M model ties Dripper at a third the size. Frontier LLMs score higher on this benchmark, near 0.90, which is the quality Pulpie approaches.
Dripper returns nothing on 135 pages. 130 are due to the page overflowing its 32k-token context window. Pulpie packs blocks into 8,192-token chunks, so page length never forces a failure.
Breaking results down by difficulty:
| Method | All | Simple | Mid | Hard |
|---|---|---|---|---|
| magic-html | 0.700 | 0.773 | 0.697 | 0.637 |
| Trafilatura | 0.619 | 0.721 | 0.619 | 0.526 |
| Pulpie Orange Small | 0.862 | 0.906 | 0.868 | 0.813 |
| Dripper | 0.864 | 0.913 | 0.865 | 0.817 |
| Pulpie Orange Base | 0.863 | 0.906 | 0.868 | 0.818 |
| Pulpie Orange Large | 0.873 | 0.914 | 0.879 | 0.827 |
Every method loses ground as pages get harder. The heuristics fall fastest, dropping 14 to 20 F1 points from simple to hard, while the encoders give up about 9 F1 points. Dripper’s performance range matches the encoders, with a gap of 10 F1 points between simple and hard pages.
Speed
Throughput by Model
20x faster than Dripper on L4, comparing Pulpie Small on the same pages.
L4 throughput, on 500 real Common Crawl pages:
| Method | Throughput (pages/sec) | Hardware |
|---|---|---|
| Pulpie Orange Small | 13.7 | L4 |
| Dripper | 0.68 | L4 |
| Pulpie Orange Base | 3.9 | L4 |
| Pulpie Orange Large | 1.3 | L4 |
Pulpie Orange Small runs 20x faster than Dripper on the same L4.
A100 throughput, same pages, GPU inference only, batched for every model:
| Method | Throughput (pages/sec) | Hardware |
|---|---|---|
| Pulpie Orange Small | 25.7 | A100 |
| Dripper | 3.6 | A100 |
| Pulpie Orange Base | 7.7 | A100 |
| Pulpie Orange Large | 3.5 | A100 |
On the A100, Pulpie Orange Small runs 7.1x faster than Dripper. The 2.1B teacher matches Dripper on speed while beating it on quality.
Cost
Cost per 1B Pages
20x cheaper than Dripper on L4, comparing Pulpie Small on the same pages.
L4 cost for 1 billion pages at $0.39/hr. Calculated using the throughputs measured above:
| Setup | Pages/sec | GPU-hours / 1B | Cost / 1B pages |
|---|---|---|---|
| Pulpie Small on L4 | 13.7 | 20,300 | ~$7,900 |
| Dripper on L4 | 0.68 | 408,000 | ~$159,000 |
| Pulpie Base on L4 | 3.9 | 71,200 | ~$28,000 |
| Pulpie Large on L4 | 1.3 | 214,000 | ~$83,000 |
A100 cost for 1 billion pages at $2.72/hr. Calculated using the throughputs measured above:
| Setup | Pages/sec | GPU-hours / 1B | Cost / 1B pages |
|---|---|---|---|
| Pulpie Small on A100 | 25.7 | 10,800 | ~$29,000 |
| Dripper on A100 | 3.6 | 77,200 | ~$210,000 |
| Pulpie Base on A100 | 7.7 | 36,100 | ~$98,000 |
| Pulpie Large on A100 | 3.5 | 79,400 | ~$216,000 |
Cheap GPUs like Encoders
The throughput gap between Pulpie and Dripper is much larger than a 3x difference in size would imply. On the A100, we measure this gap as 7.1x, and on the L4 it widens to 20x. The reason for this is architectural.
A decoder generates labels one token at a time. Each step reads the full model from GPU memory to produce a single token. Consequently, a decoder’s speed is bound by memory bandwidth. Conversely, an encoder runs one forward pass over the whole input. This dense matrix multiply is limited only by compute.
Add to the above that A100 and L4 differ more in bandwidth than in compute:
| Dimension | NVIDIA A100 | NVIDIA L4 | Ratio (A100/L4) |
|---|---|---|---|
| Memory Bandwidth | 2,039 GB/s | 300 GB/s | ~6.8x |
| Tensor Core TFLOPS | 312 | 120 | ~2.6x |
Dropping from A100 to L4 starves the bandwidth-bound decoder far more than the compute-bound encoder. This widens the throughput gap and lets Pulpie Orange Large pull ahead on L4 despite matching Dripper on A100.
The Pulpie models are on Hugging Face. Install the package:
pip install pulpieExtract clean content from raw HTML:
from pulpie import Extractor
extractor = Extractor() # defaults to Pulpie Orange Small
result = extractor.extract(html)
print(result.markdown) # clean markdown
print(result.n_main, result.n_other) # blocks kept vs droppedFor maximum quality over speed, pick a larger model:
extractor = Extractor(model="large") # "small" (default), "base", or "large"For bulk processing, the pipeline overlaps CPU preprocessing with GPU inference across one or more GPUs:
from pulpie import Pipeline, PageInput
pipeline = Pipeline(model="small")
results = pipeline.extract_batch(
[PageInput(html=h, page_id=i) for i, h in enumerate(pages)]
)All three models are built on EuroBERT (Boizard et al., 2025), use the same <|sep|> block-marker architecture, and share a tokenizer:
| Name | Hugging Face | Parameters | ROUGE-5 F1 | Notes |
|---|---|---|---|---|
| Orange Small | feyninc/pulpie-orange-small-v1 | 210M | 0.862 | Recommended |
| Orange Base | feyninc/pulpie-orange-base-v1 | 610M | 0.863 | Distilled from Large |
| Orange Large | feyninc/pulpie-orange-large-v1 | 2.1B | 0.873 | Teacher |
Pulpie Orange Small is the recommended and default model. It approaches SOTA extraction quality at one twentieth the cost and runs the fastest.
Pulpie is built by Feyn. Find us on GitHub, Hugging Face, or X.
Acknowledgements
Pulpie builds directly on the work of the MinerU-HTML and Dripper team (Ma et al., 2025). Their simplify_html preprocessing, block-level annotation scheme, and the WebMainBench benchmark are foundational to this work. We also use their Dripper 0.6B model to cross-validate our training labels. We’re grateful they released their tools and data.
References
[1]
Ma et al. "AICC: Parse HTML Finer, Make Models Better — A 7.3T AI-Ready Corpus Built by a Model-Based HTML Parser."
arXiv:2511.16397
(2025).
[2]
Boizard et al. "EuroBERT: Scaling Multilingual Encoders for European Languages."
arXiv:2503.05500
(2025).
[3]
Hinton et al. "Distilling the Knowledge in a Neural Network."
arXiv:1503.02531
(2015).
[4]
Raffel et al. "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer."
JMLR
2020.
[5]
Penedo et al. "The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only."
NeurIPS
2023.
[6]
Penedo et al. "The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale."
arXiv:2406.17557
(2024).
[7]
Li et al. "DataComp-LM: In Search of the Next Generation of Training Sets for Language Models."
NeurIPS
2024.
[8]
Soldaini et al. "Dolma: An Open Corpus of Three Trillion Tokens for Language Model Pretraining Research."
ACL
2024.
[9]
Barbaresi. "Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discovery and Extraction."
ACL/IJCNLP
2021.
[10]
Kohlschütter et al. "Boilerplate Detection using Shallow Text Features."
WSDM
2010.
[11]
Pomikálek. "Removing Boilerplate and Duplicate Content from Web Corpora."
PhD thesis, Masaryk University
, 2011.
[12]
Bevendorff et al. "An Empirical Comparison of Web Content Extraction Algorithms."
SIGIR
2023.
[13]
Shi et al. "Large Language Models Can Be Easily Distracted by Irrelevant Context."
ICML
2023.
Cite this note
@note{pulpie2026,
title = {Pulpie: Pareto-Optimal Models for Cleaning the Web},
author = {Minhas, Bhavnick and Nigam, Shreyash and Feyn Research},
year = {2026},
venue = {Feyn Field Notes}
}