|
| None | __init__ (self, *Layer|Activation architecture, Optional[int] batch_size=None, str device="cpu", str dtype="float32") |
| | __getattr__ (self, str name) |
| "FullPCN" | configure (self, float learning_rate=1e-6, float inference_rate=0.1, float feedback_rate=1e-4, float lmbda=1e-2, str optimizer="SGD", str psi_optimizer="SGD", bool use_mu_pc_scaling=False, bool use_residual_connections=False, bool use_ipc=False, bool use_epc=False, bool use_momentum=False, float momentum_beta=0.9, bool use_cross_entropy=False, Optional[int] seed=None, str dist="normal") |
| None | set_use_mu_pc_scaling (self, bool enabled) |
| None | set_use_residual_connections (self, bool enabled) |
| None | set_use_ipc (self, bool enabled) |
| None | set_use_epc (self, bool enabled) |
| None | set_use_momentum (self, bool enabled, float beta=0.9) |
| None | set_use_cross_entropy (self, bool enabled) |
| None | set_optimizer (self, str optimizer) |
| None | set_psi_optimizer (self, str optimizer) |
| None | set_learning_rate (self, float learning_rate) |
| None | set_feedback_rate (self, float feedback_rate) |
| None | set_inference_rate (self, float inference_rate) |
| None | compile (self) |
| None | randomize_weights (self, Optional[int] seed=None, str dist="normal") |
| float | train_step (self, npt.NDArray[np.float32] X, npt.NDArray[np.float32] Y, int inference_steps=1) |
| npt.NDArray[np.float32] | predict (self, npt.NDArray[np.float32] X, int inference_steps=1) |
| "FullPCN" | fit (self, npt.NDArray[np.float32] X, npt.NDArray[np.float32] Y, int epochs, int inference_steps=1, Optional[float] initial_lr=None, float decay_rate=1.0, bool shuffle=True) |
The "everything" Predictive Coding Network: every PC mechanism this
library implements, bolted onto one class as independently toggleable
extras, each OFF by default so a plain FullPCN reproduces DKPPCN
(DirectKPPCNetwork) exactly.
DKP direct feedback (Psi) is always active, same as DKPPCN, so
inference_steps=1 is a reasonable default operating point even before
any extra is enabled. The extras, each a configure() keyword:
use_mu_pc_scaling:
muPC's full Table 1 parameterization (per-layer forward scale
`a` plus unit-variance weight init), replacing plain PC's
fan-scaled init. Must be decided at configure() time: `a` is
computed from the full, now-known architecture and baked in
by compile(), so toggling it later has no effect. See
Innocenti et al., https://arxiv.org/abs/2505.13124.
use_residual_connections:
Adds a skip connection (mu += z, after activation) on every
middle hidden layer. Requires matching width between a
layer's input and output; compile() raises ValueError
otherwise. Also compile()-time-only, like use_mu_pc_scaling.
use_ipc:
Incremental PC: weights update every settling step instead
of once at the end. Safe to toggle any time after configure().
See Salvatori et al., https://arxiv.org/abs/2212.00720.
use_epc:
Error-based PC: each settling step becomes one full
forward+backward sweep through every hidden layer (the exact
PC gradient via reverse-mode AD), instead of the usual
one-hop local message -- orders of magnitude fewer settling
steps on deep networks, at a higher per-step cost. Safe to
toggle any time after configure(). See Goemaere et al.,
https://arxiv.org/abs/2505.20137.
use_momentum / momentum_beta:
EMA-smooths the settling update direction before applying it
to z, instead of applying it directly. Safe to toggle any
time after configure().
use_cross_entropy:
Softmax cross-entropy energy on the terminal layer only
(plain Gaussian energy everywhere else). Safe to toggle any
time after configure().
use_mu_pc_scaling and use_residual_connections are compile()-time
decisions: set them via configure(), not afterwards. The other four
extras are read fresh every settling step and can be flipped at any
point via their own set_use_*() method.
dtype selects which templated C++ backend (FullPCNetworkT<T>) this
instance wraps: "float32" (the default, every existing caller's
behavior unchanged), "float64" for genuine double precision, or one
of the quantized storage types ("fp4_e2m1", "posit4_0", "posit4_1",
"posit4_2"). FullPCN itself no longer subclasses the bound C++
network type -- there's no longer one single bound type to subclass
once there's a distinct one per dtype -- so it holds the chosen
instance as self._impl and forwards to it, either through an
explicit wrapper method below or (for anything not explicitly
wrapped) via __getattr__.
device="cuda" is the one exception to the dtype dispatch above:
it constructs the legacy, non-templated FullPCNetwork instead of
FullPCNetworkT<float>, and requires dtype="float32" (its only
precision).
| "FullPCN" deepity.FullPCN.FullPCN.configure |
( |
| self, |
|
|
float | learning_rate = 1e-6, |
|
|
float | inference_rate = 0.1, |
|
|
float | feedback_rate = 1e-4, |
|
|
float | lmbda = 1e-2, |
|
|
str | optimizer = "SGD", |
|
|
str | psi_optimizer = "SGD", |
|
|
bool | use_mu_pc_scaling = False, |
|
|
bool | use_residual_connections = False, |
|
|
bool | use_ipc = False, |
|
|
bool | use_epc = False, |
|
|
bool | use_momentum = False, |
|
|
float | momentum_beta = 0.9, |
|
|
bool | use_cross_entropy = False, |
|
|
Optional[int] | seed = None, |
|
|
str | dist = "normal" ) |
Configure and initialize the network.
Parameters
----------
learning_rate:
Learning rate for the forward weights W.
inference_rate:
Inference/settling rate.
feedback_rate:
Learning rate for the direct feedback weights Psi.
lmbda:
Regularization parameter.
optimizer:
Optimizer used for W.
psi_optimizer:
Independent optimizer used for Psi.
use_mu_pc_scaling, use_residual_connections, use_ipc, use_epc,
use_momentum, momentum_beta, use_cross_entropy:
See the class docstring; every extra is OFF by default.
seed:
Optional seed for weight randomization, for reproducible
runs. None (the default) uses a nondeterministic seed.
dist:
Distribution for W's init, e.g. "normal(0,1)" or
"uniform(-0.3,0.3)". The default "normal" has no "(a,b)"
and so deliberately falls back to each layer's own
existing muPC-aware default (see RandomizeWeights()).
1.18.0