Themata.AI
Follow coverage
Themata.AI
← All topics

Hierarchical networks

Latest stories covering Hierarchical networks, with summaries and links to related coverage.

1 stories · 1 reports · page 1 of 1

Browse stories ↓Search this topic

0 stories · 0 reports in the last 7 days

Latest report · Counts as of (UTC)

Models in this coverage

No tracked models are linked to this topic’s reports in the last 30 days.

    No established topics share multiple stories with this topic yet.

    Latest stories

    • Scaling Hierarchical Networks

      opinion

      29 min read

      Cartesia Research reported that its hierarchical networks, or H-Nets, learn to group raw bytes into dynamic chunks inside a language model rather than relying on a fixed Byte Pair Encoding tokenizer. The October 2026 results compare byte-level H-Nets with tokenized Transformer baselines under matched data and compute settings. In a one-stage configuration designed as a learned replacement for tokenization, Cartesia said H-Nets were 1.28 times more compute-efficient on in-domain validation loss and about twice as efficient on downstream evaluation suites. Across 15 programming languages in the training mix, the reported median efficiency gain was 2.04 times; across 10 out-of-domain languages, it was 3.10 times, reaching 5.37 times on Prolog. A larger two-stage byte-level H-Net used a 40-layer Transformer core with Mamba-2-based encoder and decoder stages and was compared with Hugging Face’s 3.08-billion-parameter SmolLM3. Cartesia reported that its 3.67-billion-parameter H-Net reached SmolLM3’s final average benchmark score after roughly 1 trillion training tokens, versus about 11 trillion tokens for SmolLM3, while matching compute per input byte within 2%. Cartesia also reported dynamic, context-dependent chunking behaviors, including grouping recurring words, formats, code identifiers, and motifs. Preliminary experiments extended the architecture to DNA and pixel-space image generation; Cartesia reported a 6–7 times compute gain for DNA sequence modeling.

      AI agentsMachine learningData efficiency
    Explore all topics →