Five notebooks, one neural network, and a question I couldn’t stop poking at: what if a transmitter didn’t care about bits at all?

Last month, I picked up a 2021 paper called DeepSC-S (Weng, Qin & Li — arXiv:2012.05369) and decided I wasn’t going to just read it. I was going to try to reproduce the entire paper and write down every place my intuition turned out wrong. This is that write-up about. Code’s on GitHub, trained weights are on Hugging Face.

The idea, stripped down

Every phone call you’ve ever made splits the job of “get my voice to you” into two completely separate problems. First, a codec (coder-decoder) such as Opus (not the LLM), AMR that squeezes your voice into as few bits as it can. Then a channel coder such as Turbo codes or LDPC that wraps those bits in redundancy so they survive a noisy connection.

DeepSC-S throws that split out. One neural network does both jobs at once, trained end-to-end against the quality of the reconstructed speech itself. It doesn’t need to get the bits back exactly right. It just needs the waveform on the other side to sound close enough. That’s the whole “semantic” pitch — the network learns to protect whatever it is about the signal that carries meaning, not whatever happens to be a 1 or a 0.

I’ll be honest, when I first read that sentence I wasn’t sure I believed it. It sounds like the kind of claim that’s true in a diagram and false in a training log. So I built it in five stages, each one proving exactly one new piece of the argument before I let myself trust the next one.

Stage 1 — turning sound into something a CNN can look at

The first surprise had nothing to do with neural networks. It was about how you even hand audio to a 2D convolution.

There’s no FFT here, no spectrogram, none of the usual audio-ML machinery. The whole trick is a reshape: take a fixed 16,384-sample clip and lay it out as a 128×128 grid, one row at a time, no overlap, no windowing.

That picture is the entire reason this works. Row-major keeps everything in a row close together in time, so when a 5×5 filter slides across the grid, it’s looking at genuinely local structure, the same way it would on a photograph. Do it column-major instead and you scatter samples that were 128 apart in time into the same neighborhood, and the CNN is staring at what looks like structured noise.

(16,384 was picked because it’s 2¹⁴, nothing deeper than that. Round numbers make for restful debugging.)

Stage 2 — does the autoencoder even work, with no noise at all?

Before I let any noise near this thing, I wanted to know the encoder-decoder pair could reconstruct speech on its own. If that fails, nothing downstream is worth debugging.

The model itself is small — about 115,000 parameters. A stem pushes the framed audio from 1 channel up to a 32-channel feature map, four SE-ResNet blocks refine it (spatial size never shrinks, only the depth axis moves), a channel encoder compresses it down for “transmission,” and a mirrored path decodes it back.

First, a sanity check: I overfit four clips for 1,200 epochs just to prove the architecture could memorize — loss went from 0.16 down to 0.00017. That confirmed the wiring was right before I trusted anything bigger. Then, on an actual train/test split, mean reconstruction quality landed around 24.5 dB SDR, comfortably past the 20 dB line people usually call “near-lossless.”

The part I actually wanted to check, though, was whether the squeeze-excite gates inside those SE-ResNet blocks were doing anything real or just decorating the architecture. Each block computes a per-channel gate value between 0 and 1, recomputed fresh on every forward pass, and multiplies it into the features before adding the residual back. It’s supposed to be an attention mechanism. So I zeroed the six highest-gated channels and separately zeroed the six lowest-gated ones and compared the damage. Killing the “important” channels cost 11.3 dB. Killing the ones the gate had already suppressed cost 13.0 dB — barely worse. In other words, the gate really is tracking something the network relies on, not just producing numbers that look plausible.

Stage 3 — adding noise means fixing power first

This is the stage where I learned that “SNR” is a meaningless number if you don’t pin signal power down first, and I mean that as someone who thought he already understood SNR.

Before anything hits the simulated channel, every batch gets rescaled so its average power is exactly 1.0. Skip that, and your SNR drifts with every training step just because the network’s output magnitude drifts, which means “10 dB” means something different on step 400 than it does on step 4,000. Once power is pinned, the noise standard deviation becomes a clean function of the SNR you ask for: σ = √(10^(−SNR/10) / 2). At 0 dB, that’s a σ of about 0.71. At 18 dB it’s down to 0.089.

The training loop samples a fresh random SNR between 0 and 20 dB on every single step, not once per run. That single choice is what makes the resulting model robust across a whole operating range instead of just good at whatever one SNR you happened to train it on — classic data augmentation, just applied to noise level instead of pixels. And because torch.randn is differentiable, gradients actually flow backward through the noise itself, from decoder through channel through encoder.

One thing I didn’t expect: at a fixed 0 dB, it’s the high-frequency detail that disappears first — consonants, sibilants, the transient stuff. Low-frequency voiced content hangs on the longest, purely because it dominates total signal power and MSE loss protects power, not fine detail. Makes sense in hindsight. Didn’t expect it going in.

Stage 4 — fading is multiplication, not addition, and that changes everything

AWGN just adds noise: y = x + w. Real wireless channels also fade — multipath propagation makes the channel gain itself a random complex number, so you get y = h·x + w instead. Rayleigh fading assumes no direct line of sight at all, pure scatter. Rician assumes a line-of-sight path plus some scatter, weighted by a K-factor — crank K to infinity and you recover AWGN, drop it to zero, and you recover Rayleigh.

The question I actually cared about here: if I train a model on one channel, does it survive a channel it’s never seen?

Two things jumped out. First, the gap between “tested on what it was trained on” and “tested on something else” is biggest at low SNR and nearly vanishes by 20 dB, which makes sense — once noise stops dominating, the fading mismatch matters less. Second, and this is the one I didn’t see coming: a model trained on Rayleigh generalizes to AWGN better than a model trained on AWGN generalizes to Rayleigh. Train on the harder channel and you get an encoding that’s robust by default. Train on the easy channel and hope it transfers, and it mostly doesn’t.

That’s a testable, falsifiable claim, and it held up again later on real data, which is the kind of thing that makes a reproduction project feel worth doing.

Stage 5 — real speech, and where the numbers actually land

Everything up to here ran on cheap synthetic data on purpose, so that if something broke, I’d never have to wonder whether it was my model or my download. Stage 5 is where the Edinburgh DataShare speech set (Valentini 2016, the standard clean/noisy speech benchmark) finally shows up.

I trained on a 2,000-clip subset of the 28-speaker clean set for 40 epochs, AWGN only, then evaluated across all three channel types without retraining.

Same shape as the toy-data result — a real gap between matched and mismatched channels at low SNR, closing up by 20 dB. I also wired in PESQ (Perceptual Evaluation of Speech Quality, the industry-standard metric that actually correlates with what a human ear judges as “sounds bad”), because SDR is just a signal-energy ratio and can’t tell you that. PESQ agreed with SDR on the channel ordering — AWGN best, Rayleigh and Rician trailing — but stayed low in absolute terms, around 1.0 to 1.35 on a 1–4.5 scale. That’s not a bug. That’s exactly what you’d expect from a model trained on a 2,000-clip subset for 40 epochs instead of the paper’s full run on the whole dataset. I want to be upfront about that rather than let a chart imply something the training budget didn’t earn.

That’s also the honest answer to “did you fully reproduce the paper”: No. I reproduced the mechanism faithfully — the architecture, the training procedure, the channel simulation — while deliberately scaling down the training budget for something that fits in a single sitting on a personal machine. Every place the paper’s text was ambiguous (kernel sizes, the SE reduction ratio, the Rician K-factor, even the choice of optimizer), I checked against the official TensorFlow implementation rather than guess, and I wrote each of those choices down instead of burying them.

The full repo, including all five notebooks with their outputs already saved so you can read them without running anything, is on GitHub. Trained checkpoints for all three channel types are on Hugging Face if you just want to load weights and start poking without training anything yourself.

If you build on any of this, or find a place where my choices diverge from something you know about the paper that I got wrong, I’d genuinely like to hear about it.

Originally published on Medium.