The Theory of Everything is an overarching hypothetical framework that would explain the physics of the entire universe in a single equation. But unifying theories that define the large-scale cosmological structure of the universe with those that describe the minuscule quantum world of the subatomic particles has been a challenge for over a century.
Figuring out such an all-encompassing theory was the dream of two legendary physicists, Albert Einstein and Stephen Hawking. But although equations that describe the universe on the largest and smallest scales have become more precise over the decades, they still don't unite to provide a complete picture of the physical world. The situation is so exasperating that some of the greatest physicists of today concede that they might not live to see it all fall into place. Hawking himself had given up on the search for a Theory of Everything before his death in 2018.
Cambridge University astrophysicist Christopher Reynolds admits that Einstein's sense of "aesthetics for the universe" might be offended by the "complex and messy" nature of current attempts to figure out the rules of the cosmos. While the iconic German-born thinker was able to encapsulate the workings of the world on the large scale, where the rules of gravity reign supreme, in the neat E = mc^2 (a simplified form of an equation that shows energy is equal to mass times the speed of light squared), things began to crumble when physicists attempted to reconcile his theory of general relativity with quantum physics, which describes the rules that govern the world on the smallest scales.
Related: Do parallel universes exist? We might live in a multiverse.
"Standard physics right now really has two legs to it," Reynolds told Space.com. "One of them is the Standard Model of particle physics. Which is a beautiful theory that explains the properties of matter. But it doesn't explain gravity, which is the other leg."
Gravity versus quantum field
The Standard Model of particle physics is the foundation of quantum mechanics that describes the world of atoms and their constituent particles such as the quarks and gluons, that make up protons and neutrons in atomic nuclei and electrons that orbit them. The Standard Model explains three of four fundamental forces that govern the natural world: the electromagnetic force that holds atoms and molecules together through the interaction of their electrically charged components, the strong nuclear force which binds elementary particles called quarks into more complex protons, neutrons and electrons (and subsequently into atoms), and the weak nuclear force responsible for radioactive decay.
These forces are a result of particle interactions, Michael Duff, an Emeritus Professor in theoretical physics at Imperial College London, told Space.com. A photon exchanged between two electrons produces the electromagnetic force, the W and Z bosons explain the strong and weak nuclear forces. But try explaining gravity by interacting particles, and you get to a point where the math goes awry, Duff said.
"According to Einstein, it's the geometry of space and time which is responsible for the gravitational force," Duff told Space.com. "You can ask yourself whether gravity perhaps could be a result of a particle called the graviton. And that works quite well to a certain degree. But when you try to make a full quantum theory of gravitons, it goes horribly wrong. Your answers, which should be finite, turned out to be infinite."
On the large cosmic scales, where the rules of Albert Einstein's general relativity match observations, quantum principles don't seem to apply. The same happens when one looks for the principles of gravity in the microworld of subatomic particles.
Duff, who has spent most of his adult life trying to reconcile the two theoretical frameworks, describes their incompatibility as "the disaster of the 21st century."
The journey to sort out this "disaster" started a hundred years ago, and Duff, now 73, admits that he may not see the day when the "rules of chess" as he calls it, are finally cracked and the Theory of Everything is complete.
"I'm not expecting it anytime soon," Duff said. "I think the key word is patience. It's going to take a long time, a lot of more research before we get there, that's my guess."
Kaluza-Klein theory and the birth of a multi-dimensional universe
Even before Albert Einstein turned his famed brain to the Theory of Everything, his contemporaries Theodor Kaluza and Oskar Klein attempted to marry his theory of general relativity with James Clerk Maxwell's theory of electromagnetism, which in the late 19th century had provided an overarching explanation for the two main forces known at that time: magnetism and the electrical force.
To make their theory work, Kaluza and Klein had to invent a world that looked very different from what we see around us. They had to add a fifth dimension to our three-dimensional space plus time. This fifth dimension, however, was curled up and microscopic, a tiny loop that we cannot see on the level of everyday life.
It was this theory that Einstein attempted to further develop into a unified field theory, which would describe all fundamental forces, including gravity, and the relationships between elementary particles in terms of a single theoretical framework without the need for quantum physics. His attempt ultimately failed. Since then things have only gotten more complicated. Electromagnetism was superseded by the more complex quantum mechanics, which Einstein, according to reports, never fully accepted as it seemed too "counterintuitive." Moreover, the number of dimensions of the universe needed for the evolving Theory of Everything (sometimes called quantum gravity) to work, has more than doubled.
Michael Duff is an Emeritus Professor of theoretical physics at Imperial College London. Duff gained his PhD in theoretical physics in 1972 at Imperial College under Nobel Laureate Abdus Salam and has spent decades researching the unifying theory of physics. He is known for his contributions to the development of quantum gravity, supergravity, string theory and M-theory. He wrote the first book devoted to M-theory, The World in Eleven Dimensions: Supergravity, Supermembranes and M-theory, which he published in 1999. He is a recipient of the 2004 Meeting Gold Medal from the El Colegio Nacional, Mexico, the 2017 Paul Dirac Gold Medal and Prize from the Institute of Physics, the U.K., and the 2018 Trotter Prize, USA.
String theory and the multiverse
The first big breakthrough since Kaluza and Klein's 1920s theory came in the 1980s in the form of String Theory. At that time, physicists, desperate to get rid of the infuriating infinite values produced by the theoretical colliding graviton particles (mentioned by Duff), proposed that elementary particles of the microcosmos perhaps weren't simple points in space but instead tiny loops of strings, which only appear pointlike to us.
"It looked for a moment maybe as though this was the answer to all our prayers," Duff said. "But soon we found that there were other problems."
Just like Kaluza-Klein Theory, String Theory didn't work in the ordinary four-dimensional universe. But it didn't work in Kaluza and Klein's five dimensions either. A universe of 10 and ultimately 11 dimensions emerged on physicists' blackboards, where not just one but six to seven dimensions had to be curled up in the invisible realm for the theory to work.
"It turns out that with String Theory you can do some fairly wonderful things," said Reynolds. "You can work out the vibration modes of these strings and then you figure out that the different vibration modes can take on the characteristics of different particles. An electron would be a string with one vibration mode, a quark would be a string with another vibration mode. You start to be able to describe different particles in nature as being different vibration modes of these strings."
So far, so good. The problem is that the way a string vibrates depends on how it's wrapped up. When mathematicians tried to calculate the number of possibilities of this wrapping-up, they arrived at astounding values.
"The number they often quote is 10 to the power of five hundred," Reynolds said. "That is a ten with 500 zeros after it. That's the number of different ways you can wrap the strings up."
Each of these wrapping combinations generates a possible universe in four dimensions, added Duff, a nearly endless plethora of possibilities one of which should represent the universe we inhabit.
"Some of them look like our universe, with the right numbers of quarks and electrons and so on, but some of them look nothing like our universe," Duff said. "And the problem we're faced with in String Theory is how do we pick the right one? Is there a right one? Because there seem to be billions of different possibilities."
By suggesting a multitude of different recipes for spacetime, the fabric of the universe, String Theory helped originate the multiverse concept, a theory of alternate universes alongside ours which may possess different physical laws than the universe we live in.
Christopher Reynolds is a professor of astrophysics at the University of Cambridge. He is a specialist in observational and theoretical high-energy astrophysics. He studies the role of black holes in the universe and relativistic phenomena that appear in the vicinity of black holes where the rules defined by the Standard Model of particle physics break.
Supersymmetry and supergravity
To make the String Theory equations work, physicists had to reconcile the behaviors of two types of particles: bosons and fermions. Quarks, the building blocks of protons and neutrons, are fermions as are electrons. This means fermions are the fundamental constituents of matter. Bosons, like photons, gluons, and W and Z bosons, on the other hand, carry the forces that hold this matter together. Both of these types of particles are characterized by their spin, which is the amount of angular momentum a particle possesses and determines which way it will travel when exposed to a magnetic field. The spin values of fermions and bosons can also exist in discrete amounts and spin is conserved for all particles, but these values are very different for these families of particles..
"Bosons have an intrinsic angular momentum which is a [whole number] like 0,1,2,3," Duff explained. "Fermions have spins in halves: spin half, spin three halves. So for many years, we thought these two [types of particles] were like chalk and cheese. We couldn't put them together."
Physicists solved this problem with the concept of supersymmetry, which assumes that each Standard Model particle has its "superpartner" in the other group. This means a bosonic superpartner for each fermion and a fermionic superpartner for each boson, with these superpartners possessing a spin number that differs by a half to its Standard Model counterpart.
Inserting supersymmetry into the equations helped string theorists to settle on 11 instead of 10 dimensions of the universe. This development pleased Duff, who, at that time, was a part of a group of theorists developing a theory called supergravity. The theory of supergravity didn't operate with strings but with what Duff describes as "membranes", and these membranes would only work in 11 dimensions.
"For a while, we were a splinter group looking at 11 dimensions and seeing where it took us," Duff said. "The string theorists were still looking at 10 dimensions and for a while it wasn't clear whether we were on the same page."
M Theory
Throughout the 1980s and 1990s, competing approaches were developing side by side. Then, in 1995 came another breakthrough when American physicist Edward Witten proposed his M Theory. M Theory, according to Duff, provided an umbrella for the various String Theory variations that existed at that time.
"At first, there were six different approaches," said Duff. "And Witten showed us that they weren't really six different theories but rather six different corners of a deeper, more profound theory that he called the M Theory."
M Theory solved many problems, Duff added. It enabled physicists to perform more exact calculations and reconciled String Theory with Stephen Hawking's black hole formula and his theory that black holes 'leak' a form of thermal radiation that would come to be known as "Hawking Radiation," evaporating as they do so.
M Theory also introduced what Duff calls the holographic principle, which states that "the gravitational world in a certain number of dimensions can be described by a non-gravitational theory that lives on its boundary, which has one dimension less," said Duff, admitting that the claim, while rather "astonishing," seems to work.
Still, the Theory of Everything is far from worked out, Duff said. Most importantly, physicists still don't know how to pick out from billions of possible string-wrapping combinations the one combination that fits our universe.
"Whether M theory is the right theory or not, we don't know, but it's the most promising candidate," said Duff. "But if it is, how long it will take us to figure out all the details is anyone's guess."
The Theory of Everything's missing pieces: Axions and dark matter
In the meantime, scientists keep looking for the missing piece of information that could plug the holes in a potential Theory of Everything. Experiments in particle accelerators such as CERN's Large Hadron Collider in Geneva, or observations of the most distant universe may one day produce the breakthrough that generations of theoretical physicists have been waiting for.
That breakthrough, Reynolds said, will most likely come from research into the nature of dark matter, the elusive invisible substance that must make up about 85% of all matter in the universe to explain the gravitational behavior of galaxies and galaxy clusters.
"There is no explanation for dark matter in the Standard Model of particle physics," Reynolds said. "There's something out there that we just have very low-fidelity data on. If we could somehow detect those dark matter particles or detect some signature of dark energy [the force driving the accelerating expansion of the universe], then that would start to really say whether the Theory of Everything is really something along the lines of String Theory or whether it's something completely different."
Astrophysicists, such as Reynolds, as well as particle physicists working with particle accelerators, have an idea of what they are looking for: a particle called axion, which is suggested as a candidate for dark matter particles and is, in fact, predicted by String Theory.
"There's a flurry of experiments right now happening that are trying to detect these axion signatures," Reynolds said.
Researchers think axions have an odd ability to convert into X-ray photons when traveling through powerful magnetic fields. X-ray observatories such as NASA's Chandra space telescope may therefore play an important role in finding out whether axions actually exist.
"We are trying to look at some of the largest magnetized systems in the universe such as galaxy clusters," said Reynolds. "You can have something like 1,000 galaxies trapped in the gravitational potential of dark matter surrounded by magnetized hot gas. And through this gas we can see X-rays generated by supermassive black holes. If axions exist, there is a probability that some of the X-ray photons will convert into axions and we will be able to see that as distortions in the light spectra that we measure."
Future missions, such as the European Space Agency's Athena mission, as well as next-generation particle colliders, may finally find the answer.
What will happen if we finally crack the Theory of Everything?
What will happen when all the pieces of the puzzle finally fall into place and we understand how our world works? Will that be the end of physics? Duff disagrees. After we learn the "rules of chess", he said, we can finally "start playing the game".
Cracking the Theory of Everything will surely lead to a flurry of Nobel Prizes. But what comes next? We will have to wait and see.
Additional resources
For a deeper dive into the quest for the Theory of Everything, read Michael Duff's articles A maverick's quest for quantum gravity or Erratum to: String and M-Theory: Answering the Critics. You can also check the book Superstring Theory, of which Edward Witten is a co-author. If you still don't have enough, you can dive into Supergravity by one of the theory's godfathers Daniel Freedman.
Bibliography
Duff, M. A maverick's quest for quantum gravity, Physics World, December 2020, IOP Publishing
https://michaelduff.weebly.com/uploads/4/4/3/0/44306005/a_mavericks_quest._physics_world_2000.pdf
Duff, M, 2013, String and M-Theory: Answering the Critics, Foundations of Physics, November 2011
https://link.springer.com/article/10.1007/s10701-012-9696-y
Green, M.B et al, Superstring theory, July 2012, Cambridge University Press
Freedman, D. Supergravity, April 2012, Cambridge University Press
Join our Space Forums to keep talking space on the latest missions, night sky and more! And if you have a news tip, correction or comment, let us know at: community@space.com.
