Gödel, Escher, Bach by Douglas Hofstadter describes and defines the concept of recursion, and discusses its applications in computer science, consciousness, art, music, biology and various other fields.
Recursion is crucial to writing computer programs in a compact, elegant way, but it also opens the door to infinite loops and irreconcilable logical contradictions.
Self-reference makes loops possible, which is great for programming. But sometimes the computer gets stuck in those loops. XKCD gives a playful illustration of how this can happen, using ducks.
We experience these infinite loops as computer crashes. The computer isn’t “stuck” when it crashes; it’s just running the same few instructions over and over.
The computer can’t break its own loops by “stepping outside of itself;” it needs an external agent to intervene, like you hitting the reset button.
The operations of our minds are also heavily recursive and self-referential. But unlike computers, we aren’t prone to getting stuck in loops, and we seem to be unfazed by logical paradoxes. Some of us even find them beautiful. 
Biological systems are especially self-similar and fractal-like.
The profoundest truths take on the quality of strange loops, GEB’s useful shorthand for recursive paradoxes. Here’s a diagram I made of the “heterarchy” of human knowledge.
Bach isn’t the only musician to use recursion and self-reference. Hip-hop and other sample-based music use it too, in the form of artists sampling their own songs within their own songs. Here are some blog posts digging into this idea.
- Biggie Biggie Smalls Is The Illest
- Eric B and Rakim
- Nas Is Like
- In A Silent Way is a remix of itself
- Self-reference in computer programming and hip-hop
- Take it to the bridge
Hofstadter also tackles the concept of emergence, the way that an intelligent mind can arise from the interaction of unintelligent component. He compares the mind to an anthill — the collective ant colony has intelligence, even though the individual ants are dumb.
Finally, the book is the best introduction to Zen Buddhist thinking that I’ve come across. Hofstadter observes that westerners are used to thinking in terms of neat Manichean categories — profound truths are unambiguously true or false. Zen prepares the mind to deal with Gödelian paradoxes, strange loops, fractals and the like.
I don’t think I’ve ever succeeded in reading GEB from cover to cover. It’s not really that kind of book. I prefer to just open to a random page and struggle with whatever concept I find there. I recommend a similar approach.