Svetoviz

FeatureAlphaDropout

https://pytorch.org/docs/stable/generated/torch.nn.FeatureAlphaDropout.html

feature_alpha_dropout_demo.py

                import svetoviz_webgpu as sv

# 1. Define Layer Parameters
# Drops entire channels in SNN-based convolutional architectures
feature_alpha_drop = nn.FeatureAlphaDropout(p=0.2)

def terminal_callback(buffer, message, images, files):
    feature_alpha_drop.train()
    output = feature_alpha_drop(input_tensor)

    # Identify channels dropped to the saturation point (~ -1.75)
    channel_min = output.view(32, -1).min(dim=1)[0]
    dropped_count = (channel_min < -1.7).sum().item()

    buffer.send_system_message(f"Input Channels: {input_tensor.shape[1]}")
    buffer.send_system_message(f"Dropped Channels: {dropped_count}")
    buffer.send_system_message("Logic: Channel maps set to SELU saturation to preserve SNN properties.")

# 2. Start the interactive session
sv.pytorch_web(module=feature_alpha_drop, terminal_callback=terminal_callback)
                    

Spatial Alpha Dropout

FeatureAlphaDropout is the spatial equivalent of AlphaDropout. It drops entire feature maps while maintaining the self-normalizing property. This is particularly useful in deep convolutional networks using SELU, where traditional Dropout2d would break the normalized distribution required for stability.

Next Section Activations

→