pyhgf.model.transformer.HybridGPT#

class pyhgf.model.transformer.HybridGPT(tok_table, pos_table, pipeline, token_part=None, position_part=None)[source]#

The full Transformer as a mixed pipeline, from token ids to logits.

The embeddings sit outside the part contract (token ids are integers, not features). By default they are frozen lookup tables held directly. Passing token_part / position_part (learning parts fed one-hot rows, e.g. built with from_embedding()) makes them learn: a lookup equals a one-hot vector times the table, so the parts see one-hot inputs and receive the error at the embedding output. The rest of the model — blocks, final normalisation, head — is one PCSequential over (batch, seq, features) arrays.

Run it with FusedPipeline, whose step and predict take batches of integer token-id sequences, (batch, seq).

Parameters:

  • tok_table (jnp.ndarray)

  • pos_table (jnp.ndarray)

  • pipeline (PCSequential)

__init__(tok_table, pos_table, pipeline, token_part=None, position_part=None)[source]#
Parameters:

Methods

__init__(tok_table, pos_table, pipeline[, ...])

init_state()

Return (pipeline_state, token_state, position_state).