Since its discovery in 2008, astronomers have been puzzled by a cosmic mystery so vexing that it has even led some to question whether the general theory of relativity – Einstein’s masterpiece theory of gravity – is wrong on cosmic scales. The trouble is that light travelling through the universe does not seem to be affected by the gravity of large structures such as galaxy clusters in the way that Einstein had predicted.

Now we have created the largest ever map of the universe’s voids – empty regions or “holes” in space – and superclusters, which are regions with more galaxies and matter than average. This has proven Einstein right, but has reintroduced another mystery.

To understand the origin of the puzzle, we need to grasp the subtle effect that gravity has on the leftover radiation from the primordial hot plasma at the birth of the universe, known as the “cosmic microwave background”. This radiation has been travelling through the universe for 13.8 billion years, before finally being picked up by instruments on the Planck satellite – allowing us to create temperature maps that have been crucial in understanding the universe. On their journey, the light particles, or photons, of this radiation have encountered both voids and superclusters in the universe.

Einstein’s theory tells us that light experiences the effects of gravity just like matter. When a photon enters an empty void, it at first loses energy due to the greater attraction of the mass behind it. After crossing the halfway point, it then gains energy again due to the pull of matter on the other side. The net effect is similar to the change in speed a runner would experience if she ran up and down an intervening hill on her route.

Distribution of galaxies where each point is an individual galaxy, colours represent their distance from Earth. Sloan Digital Sky Survey

But what if the size of the hill changed while she was running? In fact, the universe is expanding, which stretches the “hill”, reducing its height while the photon is crossing the void. The analogy with our runner would be if she had to run further uphill than downhill on the other side: she’d slow down more going up than she sped up coming back down. Similarly, photons from the cosmic microwave background passing through voids lose a tiny bit of their energy in the process, thus appearing very slightly cooler. Conversely, photons passing through superclusters appear very slightly hotter than normal. This is called the “integrated Sachs-Wolfe effect”.

However, in 2008 a team of astronomers from the University of Hawaii attempted to measure this effect. To do so, they first identified 50 individual voids and superclusters each in the distribution of galaxies in the sky, and then measured the average temperature of each type of structure using Planck’s cosmic microwave background map. Sure enough, the radiation appeared colder when seen through voids than through superclusters. However, the apparent size of the effect was more than five times larger than predicted by calculations.

This apparent drastic failure of the theory has proved very hard to understand. Alternative theories proposed to explain it have included a completely new model of the Big Bang, a different interpretation of the properties of “dark energy” – the theoretical form of energy causing the expansion of the universe to accelerate – or even an overhaul of the theory of gravitation itself.

One mystery solved – another reopened

An important step to the resolution of the puzzle was made with our new paper, published in the Astrophysical Journal Letters. My colleague Robert Crittenden and I decided to approach the question from a different angle – instead of thinking of ways to explain the original measurement, we tried to see if the effect still held in newer data.

Temperature fluctuations in void regions (left) and supercluster regions (right). Blue is the coldest and red the hottest. The patches span 20 degrees on the sky.

To do this, we used data on more than three-quarters of a million galaxies gathered by the Sloan Digital Sky Survey to build the largest ever map of voids and superclusters in the universe. This contains more than 300 times as many superstructures as used in the 2008 study. This, combined with improvements in the statistical techniques used for the measurement, allowed us to measure the tiny temperature changes in the cosmic microwave background map caused by these superstructures with unprecedented precision.

We found that the results were exactly as expected if Einstein’s theory were correct. The newer, more powerful data simply no longer support the old Hawaii result. This conclusion can be tested again when more data become available in the future, such as from the Dark Energy Survey. But for now, the puzzle appears to be no more. Our theories of the universe survive another day.

A map of the CMB temperature fluctuations over the whole sky. The unusual cold spot region is circled at the lower right.

But in resolving one problem, however, we have reopened another – to do with a very large and unusual “cold spot” in Planck’s map of the cosmic microwave background. This spot was first noticed in 2004, and its physical origin has been a puzzle, since the odds of such a large and cold region appearing in the sky by random chance are small.

One line of speculation held that it might be caused by the gravitational cooling effect of a giant supervoid. And indeed, in 2015 astronomers found evidence of the existence of a large void in the right part of the sky – which was thought to be the biggest such hole in the universe. This was viewed as settling the argument: surely this void was the proximate cause of the cold spot.

Unfortunately, more detailed calculations showed that – if our theory of gravity were correct – the supervoid that had been found was still far too small and too shallow to explain the cold spot. If something were wrong with the theory, as the Hawaii results were suggesting, both puzzles could have the same resolution.

But our results using the new map of superstructures have now spoken in favour of Einstein. So the supervoid cannot be the explanation for the cold spot.

So while we have solved one mystery, we are back to square one when it comes to the origin of the cold spot. This is a typical development in the pursuit of science. However in the long run, we are certainly moving forward at rapid speed. It’s an exciting time to be an astronomer.

Seshadri Nadathur receives funding from a Marie Sklodowska-Curie fellowship of the European Commission's Horizon 2020 programme.

This article was originally published on The Conversation. Read the original article.