Why string theory persists — despite the knotty physics

Paul M. Sutter is an astrophysicist at SUNY Stony Brook and the Flatiron Institute, host of Ask a Spaceman and Space Radio, and author of Your Place in the Universe.

String theory is a hypothetical idea that purports to be a theory of everything, able to explain the fundamental microscopic aspects of all of reality, from the forces of nature to the building blocks of all matter. It's a powerful idea, unfinished and untested, but one that has persisted for decades.

But the theory itself had rather inauspicious beginnings, employed to explain the strong nuclear force. And it wasn't very good at it.

It's in the scattering

Up until the 1960s, physicists were feeling pretty confident: They had discovered what they thought to be the fundamental constituents of matter (protons, neutrons and electrons). And they had recently accomplished the feat of unifying quantum mechanics and special relativity with what they called quantum electrodynamics (QED), which was a completely quantum description of the electromagnetic force.

But then, they started developing incredibly powerful particle colliders, and suddenly, they weren't really liking what they were finding. In these instruments, the physicists found a bunch of broken-up protons and neutrons, revealing that these particles were not fundamental at all. And what's worse, the colliders started spitting all sorts of new kinds of particles: mesons, pions, kaons, resonances, the works.

And governing them all was an apparently new force of nature: the strong force.

The tools used to develop QED were simply falling apart with this diverse host of particles popping out of the colliders. Physicists were at a loss and willing to try new ideas.

So some theorists started rummaging around in the attic, looking for any mathematical tools that might prove useful. And there they found an interesting set of ideas first proposed by Werner Heisenberg, one of the founders of quantum mechanics.

Look, I found a string!

In the early days of quantum mechanics (the first half of the 20th century), it wasn't exactly clear what would be the best mathematical approach to explain all that weirdness. In the 1930s, Heisenberg suggested a rather extreme idea: instead of taking the normal classical physics approach of 1) write down the starting positions of all the particles involved in an interaction, 2) have a model of that interaction, and 3) follow the evolution through time of those particles, using your model to predict a result.

Instead, he argued, why don't we just skip all that work and develop a machine, called the scattering matrix, or s-matrix, that immediately jumps from the initial state to the final state, which is what we really want to measure. That machine encodes all the interaction in a giant box without actually worrying about the evolution of the system.

It was a cool idea but proved too difficult for anybody to get excited about, and it died on the vine — until physicists got desperate in the '60s.

Reviving this approach to the newfound strong nuclear force, theorists extended and developed the s-matrix idea, finding that certain mathematical functions that repeated themselves were especially powerful.

Other theoretical physicists dived in, and couldn't resist the urge to give the framework a traditional interpretation in terms of time and space and following the evolution of particles. And there they found something surprising: in order to describe the strong force, it had to be carried by tiny, vibrating strings.

Faster than a speeding photon 

These strings appeared to be the basic building block of the strong force, with their quantum mechanical vibrations determining their properties in the microscopic world — in other words, their vibrations made them look and act like tiny little particles.

In the end, this early version of string theory, known as baryonic string theory for the kinds of particles it tried to explain, didn't quite cut the mustard. It was fiendishly difficult to work with, making predictions nearly impossible. It also required the existence of particles that travel faster than the speed of light, called tachyons. That was a major problem for early string theory, since tachyons don't exist, and if they did they would flagrantly violate the incredibly successful special theory of relativity.

Oh, did I mention that baryonic string theory required 26 dimensions to make sense mathematically? That was a pretty big pill to swallow, considering that the universe has only four dimensions.

Ultimately, baryonic string theory died for two reasons. First, it made predictions that disagreed with experiments. That's a big no-no. And second, an alternative theory of the strong force, involving a new hypothetical particle called the quark and a force carrier called the gluon, was able to be folded into the quantum framework and successfully make predictions. This new theory, called quantum chromodynamics, or QCD, today remains our theory of the strong nuclear force.

And as for string theory, it mostly faded into the background. It would be revived in the 1970s, once theorists realized that it could describe more than the strong force and after they found a way to get rid of the tachyon predictions in the theory. The theory still needed extra dimensions, but physicists were able to reduce the number to a more reasonable-sounding 10. And with the realization that those dimensions could be tiny and curled up below the scale at which we could directly observe it, string theory didn't seem to wacky after all.

And today, that string theory also remains, still attempting to explain the strong force — and so much more.

Learn more by listening to the episode "Is String Theory Worth It? (Part 2: Tuning the Strings)" on the Ask A Spaceman podcast, available on iTunes and on the Web at http://www.askaspaceman.com. Thanks to John C., Zachary H., @edit_room, Matthew Y., Christopher L., Krizna W., Sayan P., Neha S., Zachary H., Joyce S., Mauricio M., @shrenicshah, Panos T., Dhruv R., Maria A., Ter B., oiSnowy, Evan T., Dan M., Jon T., @twblanchard, Aurie, Christopher M., @unplugged_wire, Giacomo S., Gully F. for the questions that led to this piece! Ask your own question on Twitter using #AskASpaceman or by following Paul @PaulMattSutter and facebook.com/PaulMattSutter.

Follow us on Twitter @Spacedotcom and on Facebook.