pyhgf.model.transformer.hybrid_from_gpt#

pyhgf.model.transformer.hybrid_from_gpt(gpt, ff_parts=None, attention_parts=None, head_part=None, token_part=None, position_part=None)[source]#

Assemble a HybridGPT from a reference Equinox GPT.

Expects the attribute layout of the reference model in docs/source/notebooks/0.8-Transformers.ipynb: tok_emb, pos_emb, blocks (each with attn.wq/wk/wv/wo, ff.fc1/fc2, n1, n2), norm_f, and head. Every slot is frozen at the given model’s weights unless a learning part is supplied for it.

With all slots frozen, the assembled pipeline computes exactly the same function as the Equinox model — the transplant wiring gate. Supplying parts (e.g. transplant converters) turns the corresponding slots into PyHGF learners while everything else stays pinned — from the single-component swap experiment up to a model whose every weight learns through PyHGF, with only the weightless calculations (normalisations, attention mixing) frozen.

Parameters:

  • gpt – The Equinox reference model whose weights are transplanted.

  • ff_parts (list | None) – Optional list of parts, one per block, replacing the frozen feed-forwards.

  • attention_parts (list | None) – Optional list of dicts, one per block, with keys "wq", "wk", "wv", "wo", replacing the frozen attention weight tables; or a "wqkv" key carrying one fused part for the stacked [q | k | v] projections (e.g. built with from_linears()), alongside an optional "wo".

  • head_part (PCModule | None) – Optional part replacing the frozen output head.

  • token_part (PCModule | None) – Optional parts replacing the frozen embedding tables (fed one-hot rows — see HybridGPT).

  • position_part (PCModule | None) – Optional parts replacing the frozen embedding tables (fed one-hot rows — see HybridGPT).

Returns:

The assembled mixed pipeline.

Return type:

HybridGPT