Deep4Net split_first_layer=False is broken by unconditional CombinedConv init/remapping #1206

Summary

Deep4Net(split_first_layer=False) still builds the unsplit first layer as self.conv_time, but the post-CombinedConv initialization and legacy checkpoint mapping now assume self.conv_time_spat exists unconditionally.

This means the documented non-split path is broken at construction time, and its legacy conv_time.* checkpoint keys would also be remapped to a module that does not exist.

Root cause

Before the CombinedConv refactor (975c02e3ce74b09fde1e5c79041c78119fce145a), Deep4 initialized self.conv_time for both modes and only touched self.conv_spat when split_first_layer=True.

After the refactor, the split path correctly creates self.conv_time_spat = CombinedConv(...), but initialization was changed to:

init.xavier_uniform_(self.conv_time_spat.conv_time.weight, gain=1)

outside the split_first_layer branch. The current legacy-key mapping similarly rewrites conv_time.* -> conv_time_spat.conv_time.* for both modes even though the unsplit model still owns conv_time.* directly.

Expected behavior

• split_first_layer=True: initialize/remap the merged temporal+spatial module.
• split_first_layer=False: initialize self.conv_time directly and keep legacy conv_time.* keys on that module.
• classifier key migration remains unchanged in both modes.

Proposed regression coverage

1. construct and run Deep4Net(..., split_first_layer=False);
2. verify the unsplit first convolution is initialized and produces a valid forward pass;
3. load a legacy-style state dict where conv_time.* remains unsplit while conv_classifier.* is migrated to final_layer.conv_classifier.*.

I can send a focused patch that only branches the CombinedConv-specific initialization/remapping and leaves the default split-first-layer behavior unchanged.