neurai.finetune.tuning.lorax package#

Submodules#

class neurai.finetune.tuning.lorax.transform.LoraWeight(w, a, b, alpha=1.0, shape=None, dtype=None)#

Bases: ImplicitArray

The weight of lora node.

Parameters:
  • w (ArrayValue) – The param of node.

  • a (ArrayValue) – The low-rank matrix.

  • b (ArrayValue) – Another low-rank matrix

  • alpha (float) – The scaling factor of low-rank matrix

  • shape (Optional[Sequence[Union[int, Any]]]) –

  • dtype (Union[Any, str, dtype, SupportsDType, None]) –