[Zephon]

Recent 40k background fills in some of the previously empty space between the Horus Heresy and the Age of Apostasy with, among other things, the Nova Terra Interregnum. The 'Ur-Council' ruled much of the Segmentum Pacificus for 900 years before the inevitable retribution, in direct challenge to the High Lords of Terra.

Astropaths are produced by the Soul-binding ritual on Terra, which as the the seat of the Emperor has a monopoly on astropath production. The astropaths must then be shipped out to Imperial worlds (Terra's 'soul' export). Astropaths aren't immortal, and all are going to lose some years to detection, storage, Black Ship transit to Terra, training for Soul-binding and outward transit.

So how could the Ur-Council maintain anything approaching a space empire for more than about 100 years? Any rebellion against the rule of Terra cuts off the supply of the astropaths necessary to rule any large volume of space.

[MarcoSkoll]

I think this can be seen more generally - how does any non-Imperial faction maintain long distance communication?

Obviously many aliens have their excuses - Eldar have the webway and frakking powerful psykers, Necrons are hugely technologically advanced and Orks wouldn't really care.
For Chaos, we can easily argue the warp did it, but the interesting one is pre-Imperial human civilisation. Although the Emperor has been around since about the 9th Millennium BCE, he wasn't exactly making himself known at the time.

My guess? Ruling out there being some manner (forbidden or otherwise) of producing a poor man's substitute for an astropath, its possibly it's simply a case of post riders.

It wasn't until the introduction of semaphore networks in the late 18th century that complex messages could start to travel long distances significantly faster than a human. (But even so, it wasn't really until the electrical telegraph that things really became practical. Semaphore doesn't work without line of sight, so no communicating across the Atlantic, sending messages at night or in bad weather). Significant human empires nonetheless predate this time - Genghis Khan, Alexander, the Romans, etc.

Now, obviously, astropathic messages have a speed advantage (and they're much harder to intercept) which would give the Imperium a notable benefit when planning a war... but to be fair, when planning a war on a segmentum level, if you're not playing several months ahead anyway, your troops probably aren't going to be in the right place regardless.

[Zephon]

Pre-Imperial humanity has a few excuses for operating a post ship system: warp travel seems to have been safer during the Dark Age of Technology - perhaps a lower average psychic strength made human souls and ships less tasty to daemons? Daemonic possession of psykers only became a problem later. Perhaps the lack of warp/real-space overlap meant the warp itself was more stable during that time? Or maybe better maintained Geller field technology? For whatever reason, the Dark Age humans never built an Astronomicon equivalent - lack of need or lack of ability? Perhaps the huge ships surviving from the Dark Ages are big because they just took a very long time to get anywhere and needed lots of supplies?

The Dark Age humans may have avoided the problem of communication in an galactic empire by simply not bothering to have one. There is no real evidence the Dark Age worlds were united by anything except trade and the STC system. Large scale military operations and long-distance communication may have been beyond their abilities and interests. Though I assume the Dark Age colonists must at least have had to deal with the odd Ork Waaagh! They don't seem to have clashed with the pre-Fall Eldar (decadence already setting in?).

[Alyster Wick]

There are a number of possibilities here. Better technology that has since been lost, more stable gellar fields making warp travel more reliable and using a ship network to transport messages across vast distances of space to regional hubs (which used less-complicated technology to send messages in-system through more traditional routes), non-Imperial human civilizations could be allied w/ aliens utilizing other means, tech that boosted/protected psychers enough to approximate astropaths (again, lost tech, kind of a running 40K theme), or some combination there of.

It's a great question that raises lots of possibilities, but I'd say it's far from impossible. An attractive option is just that there wasn't really a huge, galaxies-spanning empire. There could have been a number of empires spread across systems limited by communication itself (with some limited contact made possibly by travelers).

[Dorn]

Sometimes I think a lot of fluff might also be seen as imperial propaganda. (This is at least one way to explain parts of fluff, which just seem totally unrealistic or are contradicting other vital parts of fluff. I see most of the named Character-Fluff as propaganda or legends.)
In my humble theory it would seem quite resonable for the Imperium to create a myth, that only Terra can create astropaths. Regulating this vital part of interstellar communication, will effectifly sabotage any possible rebellions.
Maybe there are some other ways to reshape the minds of astropath? Maybe this would have helped the Ur-Council in their ruling, but the information has been lost again / possibly purged from the records by the holy Inquisition?

Yet this is just a theory and has no basic in offical GW fluff... Nonetheless would it be interesting to hear your thoughs on this.

[Zephon]

That's a good point - whatever the truth of astropath creation, the majority of the Imperium (even the astropaths themselves) probably believe the process takes place on Terra, with everything that entails.

As to other means: Pawns of Chaos had a lost Imperial colony with a sort of DIY astropath. They weren't terribly effective, but they did manage to pick up some Imperial communications. I can't remember if they actually had transmission abilities, but it was a plot point that people thought they might have. Pawns of Chaos is a Heretic Tome on Black Library, but it's also my favorite 40k story and I'll follow that take over anything else.

A friend of mine has a theory that a sufficiently large choir of astropaths could channel the Astronomican to create more astropaths, perhaps allowing major fleet bases and other communication hubs to crank out a few.

Also, there is plenty of material in Dark Heresy that can be read as indicating that a whole lot of data in the Imperium is actually carried by ships: Marco's post riders. Astropath messages must have a limit to the information they can carry, so it would be sensible to limit them to sending urgent stuff. Copies of the Book of Law and tithe assay paperwork can be handed over to passing Chartists or Navy patrols.

[TallulahBelle]

The stories being propoganda is actually Canon it's discussed in the third ed rulebook that any story told is true to the writer and the reader but there are many truths.