ATCNet does not expose source-code convolution/TCN regularization #1061

Summary

ATCNet in Braindecode matches many source-code architecture choices, but it does not expose or apply the official implementation's convolution/TCN regularization and max-norm constraints.

This matters for reproducibility of the official ATCNet result: the Keras implementation hard-codes:

dense_weightDecay = 0.5
conv_weightDecay = 0.009
conv_maxNorm = 0.6

and applies kernel_regularizer=L2(...) plus kernel_constraint=max_norm(...) to the convolutional stem and TCN convolutions.

Where to patch

braindecode/models/atcnet.py.

Relevant places:

• _ConvBlock: conv1, conv2, conv3 currently do not apply source max-norm constraints.
• _TCNResidualBlock: conv1, conv2, and optional reshaping_conv currently do not apply source max-norm constraints.
• ATCNet.__init__: no public parameters expose conv_weight_decay, dense_weight_decay, or conv_max_norm.

Evidence

Official ATCNet source:

dense_weightDecay = 0.5
conv_weightDecay = 0.009
conv_maxNorm = 0.6

The convolution block uses:

kernel_regularizer=L2(weightDecay)
kernel_constraint=max_norm(maxNorm, axis=[0,1,2])
depthwise_regularizer=L2(weightDecay)
depthwise_constraint=max_norm(maxNorm, axis=[0,1,2])

The TCN block uses:

kernel_regularizer=L2(weightDecay)
kernel_constraint=max_norm(maxNorm, axis=[0,1])

Braindecode currently constrains only the final MaxNormLinear classifier via max_norm_const, which is not equivalent to the source's conv/TCN constraints and L2 penalties.

Suggested implementation direction

There are two separable pieces:

1. Model-side constraints

   • Add optional conv_max_norm_const: float | None = None.
   • When set, apply a max-norm parametrization or forward-time clamp to _ConvBlock and _TCNResidualBlock convolution weights.

2. Training-side L2 regularization

   • Either expose helper methods returning source-faithful optimizer parameter groups, or document a recipe for using optimizer param groups:
     • conv/TCN weights: weight_decay=0.009
     • dense/final weights: source Keras uses L2(0.5)
     • biases/BN/etc.: no decay unless intentionally configured

Suggested regression tests

• Instantiate ATCNet(..., conv_max_norm_const=0.6) and assert conv/TCN kernel norms are clamped after a forward pass.
• Assert a helper such as source_optimizer_param_groups() returns separate parameter groups for conv/TCN weights and final/dense weights.

This can be added without changing current defaults, preserving backward compatibility while enabling source-faithful replication.