Predictable unpredictability.
The brain is a pattern-recognition machine. We like repetition and symmetry because they engage our pattern-recognizers. But we only like patterns up to a point. Once we’ve recognized and memorized the pattern, we get bored and stop paying attention. If the pattern changes or breaks, it grabs our attention again. And if the pattern-breaking happens repetitively, recursively forming a new pattern, we find that extremely gratifying.
Good breakbeats are just complicated enough to challenge our pattern recognition ability without totally overwhelming us. Repeating a complex and unpredictable rhythm in a simple, predictable structure, and then sometimes breaking that structure, holds our attention without completely dominating it. A good breakbeat ties the room’s attention together while still leaving enough bandwidth for people to dance, rap, sing, socialize or daydream. Breakbeats are good for social music because your attention can wander away to the other people with you, and then easily pick the thread back up.
For further reading about how music is defined by the limits of our attention spans, I recommend Music And Memory: An Introduction by Bob Snyder.
Also, I’ve written a few blog posts about programming your own beats: