IT is all around you, yet you cannot feel it. Its effects may have lit up the Universe in the big bang but today just light up your office. It is the source of everything, yet is nothing. Such are the paradoxical features of one of the hottest topics in contemporary physics - the vacuum. It is proving to be a wonderland of magical effects: force fields that emerge from nowhere, particles popping in and out of existence and energetic jitterings with no apparent power source.

Many researchers see the vacuum as a central ingredient of 21st-century physics. "We now know that the vacuum can have all sorts of wonderful effects over an enormous range of scales, from the microscopic to the cosmic," says Peter Milonni of the Los Alamos National Laboratory in New Mexico. Some even contemplate the prospect of harnessing the vacuum's bizarre properties to provide an apparently limitless supply of energy.

The vacuum's miraculous properties all stem from a combination of quantum theory and relativity. As Werner Heisenberg showed almost 70 years ago, the mechanics of the subatomic world mean that an uncertainty is attached to any measurement of physical properties such as energy. This uncertainty manifests itself in random, causeless fluctuations in energy: the larger the fluctuation, the shorter the time it survives.

Thanks to Einstein's famous equation E = mc², Heisenberg's uncertainty principle also implies that particles can flit into and out of existence, their duration dictated only by their mass. This leads to the astonishing realisation that all around us "virtual" subatomic particles are perpetually popping up out of nothing, and then disappearing again within about 10^-23 seconds. "Empty space" is thus not really empty at all, but a seething sea of activity that pervades the entire Universe.

Relatively fluid

Such an image is worryingly reminiscent of the ether — a discredited idea that bedevilled physics until the beginning of this century. But Einstein's special theory of relativity showed that physics works perfectly well without this peculiar, all-pervasive fluid, which was supposed to be the medium through which light and other interactions travelled from place to place. This does not mean that a universal fluid cannot exist, but it does mean that such a fluid must conform to the dictates of special relativity. The vacuum is not forced to be mere quantum fluctuations around an average state of true nothingness. It can be a permanent, nonzero source of energy in the Universe.

This has cosmic consequences. Special relativity demands that the vacuum's properties must appear the same for all observers, whatever their speed. For this to be true it turns out that the pressure of the vacuum "sea" must exactly cancel out its energy density. It is a condition that sounds harmless enough, but it has some astounding consequences. It means, for example, that a given region of vacuum energy retains the same energy density, no matter how much the region expands. This is odd, to say the least. Compare it with the behaviour of an ordinary gas, whose energy density decreases as its volume increases. It is as if the vacuum can draw on a constant reservoir of energy.

But there is more. One of the key features of Einstein's general relativity (GR) theory is that mass is not the only source of gravitation. In particular, pressure, both positive and negative, can also give rise to gravitational effects. If the vacuum has a permanent (positive) energy density, it must be balanced by a negative pressure (a tension). According to GR, this must give rise to a repulsive gravitational effect. This feature of the vacuum lies at the heart of perhaps the most important new concept in cosmology of the past decade: cosmic inflation. Developed principally by Alan Guth at MIT and Andrei Linde, now at Stanford, the idea of cosmic inflation arises from the assumption that the very early Universe was packed with unstable vacuum energy whose "antigravitational" effect expanded the Universe by a factor of perhaps 10⁵⁰in just 10^-32 seconds. Then the vacuum energy died away, leaving random fluctuations whose energy turned into heat. Because energy and matter are interchangeable, the result was the matter creation we now call the big bang.

At a stroke, inflation solves a number of problems that had troubled cosmologists. For example, it explains the apparent coincidence that the Universe we see today seems to be teetering between expanding for ever and collapsing. Cosmic inflation would have "flattened out" the initially highly curved surface of the Universe, and according to calculations based on GR this would have led to the amount of mass-energy that was formed being just enough to allow the Universe to escape from its own gravity and expand for ever. The behaviour of the vacuum 15 billion years or so ago thus holds the key to the future fate of the Universe.

But convenient as this is, most cosmologists would like the vacuum to have packed up its bag of tricks and disappeared once it had inflated the Universe. One reason is aesthetic. If the vacuum amounts to anything more than random fluctuations about true emptiness in today's Universe, an extra term has to be added to GR, and nobody is in a rush to make GR even more complicated.

But some reseachers are coming up with evidence suggesting that something may be missing from GR in any case. Last autumn, teams led by Michael Pierce of Kitt Peak Observatory in Arizona, and Wendy Freedman of the Carnegie Institute of Washington's observatories in Pasadena, California, both announced findings that put the age of the Universe at around 8 billion years. This was embarrassing, because there is sound evidence that some stars in our Galaxy are around twice this age.

One way out of this bind would be a vacuum state that did not vanish after inflating the Universe. Perhaps a tiny remnant of it persists, providing a gentle unseen "push" to the contents of the Universe. This would boost the speed at which galaxies race away from each other, and give the impression that the Universe as it is now is nearer to the big bang state — and thus younger - than it really is.

Vacuum energy can do more, however. Though inflation predicts that the density of mass-energy in the Universe is right on the borderline between expansion and collapse, astronomers have only found between 10 and 20 per cent of the required mass. So where is the rest? This is another problem that a remnant nonzero vacuum may solve. By Einstein's equation, an energy density is equivalent to a mass density, so vacuum energy could account for some — perhaps most — of the missing mass.

Some cosmologists, notably George Efstathiou at Oxford University, estimate that for vacuum energy to solve these problems it would have to amount to 80 per cent of the mass-energy of the Universe. But does it? Chris Kochanek of the Harvard-Smithsonian Center for Astrophysics in Cambridge, Massachusetts, says that observations of gravitational lensing show that it can't. Lensing is the phenomenon that occurs when light on its way to us from a very distant source such as a quasar passes by a galaxy, and is bent by the galaxy's gravity. This creates multiple images of the quasar. Astronomers have been surveying the sky for these effects, and counting how many lensed objects they see out to a specific distance. If some cosmic vacuum energy still exists, its antigravitational effect should expand the volume of space encompassed by a given distance and thus boost the number of gravitational lenses caught by the surveys.

Kochanek calculates that for vacuum energy to account for the required 80 per cent of the cosmic mass-energy, about 15 gravitational lenses should have been seen by current surveys. In fact, only 6 have been found. This, he says, translates to odds of around 10 to 1 against vacuum energy being more than about half of what is needed by cosmologists. Those cosmologists who support the idea of vacuum energy say that the lensing observations could have been affected by dust obscuring distant galaxies. But Kochanek says that this won't do, as all such fixes lead to inconsistencies elsewhere. With so much riding on the outcome, however, the debate looks set to continue.

While cosmologists and astronomers wrangle about lensing, physicists have been looking at the possibility that the vacuum could answer more down-to- earth questions. The most intriguing centres on inertia — the property of matter that makes heavy things hard to get moving, but once moving, hard to stop. Inertia is so familiar that its attributes seems beyond question, but they have perplexed scientists of the calibre of Einstein and Richard Feynman. If an object is at rest, or moving at constant velocity, its inertia stays hidden. But try to accelerate it and inertia suddenly rears its head, fighting against the change in velocity. This is summed up in Newton's second law of motion: F = ma, force equals inertia times acceleration.

But where does the inertia come from? Einstein believed that it was somehow induced in objects whenever they accelerate relative to the rest of the Universe, though quite how this interaction worked he never made clear. Now a group of American researchers has put a new gloss on Einstein's idea: instead of acceleration relative to the distant stars, they believe that inertia is generated by acceleration through the vacuum.

They base their idea on an esoteric quantum vacuum effect first discovered in the mid-1970s by Paul Davies, now at the University of Adelaide, and independently by William Unruh of the University of British Columbia. The Davies-Unruh effect predicts that if you accelerate through it, the usually uniform vacuum state turns into a tepid sea of heat radiation from your point of view if you accelerate through it. Two years ago, this triggered a thought in the minds of Bernhard Haisch of the Lockheed Solar and Astrophysics Laboratory in Palo Alto and, independently, Hal Puthoff of the Institute for Advanced Studies at Austin, Texas. Both wondered if, like the heat radiation, inertia is a product of acceleration through the vacuum.

Joining forces with Alfonso Rueda, a theorist at California State University, Long Beach, Haisch and Puthoff last year came up with a new version of Newton's second law. Again, it has F for force on the left-hand side, and a for acceleration on the right. But in place of M, their version featured a complex mathematical expression tying inertia to the properties of the vacuum. It implies that fluctuations in the vacuum give rise to a magnetic field through which all objects move. If the object accelerates, its constituent particles feel the grip of this magnetic field, whose resistance manifests itself as inertia. The larger the object, the more particles it contains and the stronger the reluctance to undergo acceleration.

If the theory fits ...

It is a neat idea, though it is not without its critics. Haisch and his colleagues had to deal with a problem familiar to every theorist trying to understand the vacuum, which is that estimates of the effects of vacuum energy inevitably end up having to add together all the frequencies of fluctuation that contribute to the total vacuum energy. The trouble is that some frequency limit has to be imposed, otherwise the result is an infinitely energetic vacuum. Worse still, all sensible guesses as to what the frequency cutoff might be still lead to ludicrously high values, as much as 120 orders of magnitude out of kilter with the limits set by observations of distant galaxies. As Nobel prizewinning physicist Steven Weinberg of the University of Texas puts it, "This must be the worst failure of an order of magnitude estimate in the history of science."

This problem has prompted some theorists to search for a mechanism that forces the vacuum energy to be precisely zero, while Haisch and his colleagues have tried resorting to a rather obscure theory of gravity which was sketched out in the late 1960s by the Russian physicist Andrei Sakharov. According to Sakharov's theory, the vacuum has no gravitational effects. But Milonni, for one, is not impressed with the way Haisch has applied the theory to vacuum energy.

Important as this wrangle is, it pales into insignificance compared with another consequence of the link between the vacuum state and inertia. By altering the vacuum state it might be possible to alter the inertia of objects. This is the stuff of science fiction, though as Haisch points out, "History is full of impossibilities turned into technologies, from flying to splitting atoms". He stops short of talk about spacecraft powered by vacuum energy, which "switch off" their inertia when they want to move on. "It might only prove possible to modify inertia on the atomic scale, but not the macroscopic scale," he says.

In the meantime, Haisch and his colleagues are concentrating on building up solid observational support for their theory. Later this year, The Astrophysical Journal will publish research by Rueda, Haisch and Daniel Cole of IBM in Vermont that suggests that the vacuum plays a key role in creating structure in the Universe. (Ap. J. article now online) They claim that the vacuum accelerates charged particles, sweeping them up to form concentrations of matter surrounded by vast cosmic voids. The formation of structure in the Universe is one of the oldest mysteries of cosmology, so it would be a feather in the cap of the theorists if the vacuum proved to be the missing ingredient.

But the most tantalising idea to emerge from these developments remains the prospect of manipulating the vacuum. The idea originated in 1948, when Hendrick Casimir of the Philips Laboratory in Eindhoven, Holland, made a startling prediction. Bring two perfectly conducting flat plates close to each other, he claimed, and a force will appear between them, pushing them closer together. That force, he said, was the result of the flat plates cutting off the space between them from the seething sea of the vacuum around them. It was as if the rest of the vacuum was hammering on the plates, trying to get in and thus forcing them together.

Nine years later, M. J. Sparnaay, also at Philips, verified Casimir's startling prediction. The effect is, however, incredibly feeble, amounting to a pressure of just one hundred-millionth of an atmosphere on plates held a thousandth of a millimetre apart. It may be unfamiliar, but it can be seen in the forces within liquids and gases (see "A brief history of the vacuum").

Though no one has the faintest idea how to boost the Casimir effect to a useful size, its existence has prompted some theorists, including Cole and Puthoff, to look at ways of putting the vacuum to technological use. In research published 18 months ago in Physical Review, they pointed out that as plates are drawn together by the Casimir effect they develop kinetic energy that turns into heat when the plates finally collide. They went on to look at exploiting this effect by imagining a vacuum "engine" consisting of large numbers of colliding plates. Astonishingly, their calculations showed that such an engine could indeed extract energy from the bottomless well of the vacuum. There wouldn't be much energy to play with. "Optically polished square-metre plates collapsing to one micron spacing would yield half a nanojoule, and even if the collapse took place in a millisecond, that's only half a microwatt - not much to write home about," admits Puthoff. "That's why you would need microscopic, throwaway systems running at high rate." Quite what form they would take, no one yet knows.

The solution to the cosmologist's nightmare, the explanation of inertia and the cure for the world's energy crisis? The vacuum is in danger of becoming everyone's answer to everything. But it seems a safe bet that the vacuum theorists are likely to come up with some big surprises over the coming years. The philosophers were right: nature does abhor a vacuum. Scientists of the next century may well come to love it.

A BRIEF HISTORY OF THE VACUUM

UNTIL about a century ago, the vacuum was just a vague philosophical concept. In the 17th century, for example, example, Descartes came up with the decidedly dubious argument that it was impossible to have a vacuum - that is, nothing - separating two particles, as the particles would by definition not be separated at all.

It took the advent of quantum theory, with its concept of energy coming in discrete packets, to cut through such word games. Hints that there was more to the vacuum than its name suggests first emerged as long ago as 1911, during research by Max Planck, the originator of the quantum concept.

Planck found that one of his equations for the energy of a hot body had a term in it that did not depend on temperature. In other words, even at absolute zero the body would have some residual energy. Other researchers, including Einstein, found similar terms popping up in their own investigations. This seemed bizarre, for where could this energy come from?

So physicists began to look for experimental evidence for the existence of this "energy from nowhere". In 1925, the American chemist Robert Mulliken found it, in the spectrum of boron monoxide. Analysing the frequency of its spectral lines, he discovered a slight shift, the energy for which had seemingly come from "nowhere".

Two years later, Werner Heisenberg in Germany put this "energy from nowhere" on its modern foundations with his uncertainty principle. This shows that even empty space is seething with activity, and the effects of this activity crop up in the most surprising places. For example, vacuum energy fluctuations cause random "noise" in electronic circuits, and this puts limits on the level to which signals can be amplified. Van der Waals forces, the feeble attractive forces that allow real gases to be turned into liquids, also come from distortion of vacuum energy by molecules.

This same vacuum energy also explains why cooling alone will never freeze liquid helium. Unless pressure is applied, vacuum energy fluctuations prevent its atoms getting close enough to trigger solidification. Even fluorescent strip lighting relies on the causeless, random energy fluctuations of the vacuum state. When atoms of mercury vapour are excited by the electrical discharge in the tube, their spontaneous emission of photons is triggered by vacuum fluctuations knocking them out of their unstable energy state. Every time you switch on your office lights, you are seeing an effect that physicists now think could hold the key to the big bang.