Received 14 December 2015; accepted 2 May 2016; published 5 May 2016
Whether photon and graviton possess nonzero rest masses is one of the most fundamental questions which have been actively examined during last decades both theoretically and experimentally in the lab and in the space  -  .
Contrary to Proca equations  , uniquely and undoubtedly generalizing Maxwell ones for finite range, the massive gravitation is far from its end  -  . Different theories of gravitation predict different outcomes of the same experiments, henceforth the upper bounds of the graviton mass will depend on the specific choice of such theory. We’ll consider phenomenon of breakdown of the screening effect in massive electrodynamics  and in finite-range theory of gravitation of Freund, Maheshwari and Schonberg  , and Logunov   .
When can one anticipate an appearance of nonvanishing massive electromagnetic or, correspondingly, gravitational fields if the usual massless fields are absent (screened) in that situation? For instance, it is inside the spherically-symmetric shell. It is well known, that there is no electromagnetic field in the empty charged metal conductor, having compact (in simplest case―spherical) form  . Therefore, the Lorentz force acting on the test charge is equal to zero as well as its acceleration. If the gravitons have no rest mass, then according to Birkhoff theorem, inside the massive sphere the space-time is the Minkowski one, with the acceleration of the test bodies vanishing and the shell’s gravitation field being “screened”. This is clearly not the same case as electromagnetic screening, rather it is the consequence of the spherical symmetry. Nevertheless, such shielding would be broken for finite-range gravitation. Therefore in massive case one can expect that the test charge inside the charged shell and the test mass inside the massive sphere have to move with acceleration proportional (in first approximation) to the squared mass of the photon and graviton correspondingly. As we shall see, the formulas in both cases have the same form.
We’ll consider this effect and estimate the possibilities of its observation. In particular, we’ll show that the mass of the graviton will contribute to the “Hubble constant” of the galaxies flow in Local Group. It constraints the cosmological quintessence parameter in massive relativistic theory of gravitation (RTG)   .
2. Empty Shell as the Photon Mass Detector
If the electromagnetic field has the finite range, where is the photon mass, then Maxwell equations will have the Klein-Gordon form, what was first noticed by A. Proca  . In arbitrary coordinates these equations are
where―Minkowski metrics,―vector 4-potential of the electromagnetic field,―4-current,―covariant derivative with respect to metrics. Throughout this work, we adopt the following units conventions.
Consider the solution of Equation (1). Let’s note―density of the surface charge,―ra- dius of shell. Then Equation (1) for the scalar potential yields
where―Laplace operator in spherical coordinates.
The outside solution of (2) has Yukawa form
Whereas the inner solution is
Since at the sphere surface the scalar potential is continuous, then at the origin one has
Therefore for the electric field and Lorentz force F inside the shell one obtains
According to modern evaluations  -  the upper limit of photon mass is very small, so for the laboratory scale. It yields
The force (7) is directed towards the origin for the same signs of charges and it is increased as the particle comes to the surface (weak confinement). This force less than the Coulomb one by factor. The test charge will go to the shell, when the signs of q and Q coincide. In this sense the charges of the same signs are attracted. When the signs are different, the “repulsion” takes place, making the test charge to oscillate around the origin like the plummet on the Hook spring with frequency. Cavendish-type experiment searching deviation from the Coulomb low put the upper bound and  .
The question concerning possibility of such direct detection of photon mass in the lab or in the space, where there are no free big charges, is out of the frameworks of the paper.
The density of energy of such electromagnetic field, i.e. 00-component of symmetric energy-momentum tensor
inside the cavity is almost constant and has the order of magnitude:
Both the solution (4)-(7) and stress-energy tensor for massive electrodynamics have the correct limit.
In the following section, we’ll demonstrate that there is a full analogy for the graviton of the mass m case?test body inside the spherical massive shell is no more at rest, with the force being proportional to. Contrary to the electromagnetic case of the same sign charges, the test particle will be accelerated towards the shell surface.
3. Empty Shell as the Graviton Mass Detector
Let’s consider the test body inside the thin spherically-symmetric perfect-fluid shell, keeping static by virtue of some external pressure. The origin of the pressure is undetermined in the frameworks of our task. In classical mechanics (with Newtonian inversed squared distance force) the test body is at rest inside the massive shell. The result keeps also valid for the exact solution of the gravitational field equations  . If one considers massive gravitation case, then the test body will no more be at rest in the cavity in close analogy with nonzero acceleration of the test charge in massive electrodynamics. Such cavity can be the detector of the mass of graviton. Both the sign and the value of such acceleration will be calculated in the paper.
Let’s find the gravitational field created by the thin spherically-symmetric massive shell. Using standard coordinates in spherically-symmetric case one has
where r―radius in Minkowski space, W―Schwarzschild radius,―Riemannian space-time metrics,― Minkowski metrics.
Gravitational field () equations can be written in the form, analogous to Maxwell electrodynamics  -  :
where―covariant derivative with respect to,―stress-energy tensor for perfect-fluid shell, counteracting gravitational contraction by virtue of the nonradial pressure:
―symmetrical Hilbert stress-energy tensor of gravitational field:
Equations (11), (12) can be rewritten in more conventional form:
In our case from (9), (10), (13), (14), (15) one obtains
We shall solve Equations (16)-(18) in linear in graviton mass (which is extremely small:    ) approximation. In zero order () these equations are significantly simplified:
The mass density of the thin shell is given by the -function:
From (19), with taking into account conditions at the infinity, we obtain the external solution (, where―shell radius):
Constant C will be determined further by matching with internal solution.
Let’s consider the internal solution. There is no singularities at origin, therefore from (23)- (25) we get
Thus in zero order approximation () the gravitational field inside the shell is constant and equals to
In Cartesian coordinates is also diagonal and homogenious:. Therefore the gravitational force for such field vanishes.
In the strong field limit, when the radius of the shell goes to Schwarzschild horizon, we get
In the weak field limit, the expansion up to accuracy yields:
It’s easy to show, that the function (24) is monotonously increasing and strictly positive.
Let’s study case. Solving Equations (16)-(18) in linear in approximation, after some calculations we can find finally:
Then, one finds the value of gravitational force acting on the test particle in the cavity using geodesical equations in Riemannian space:
In nonrelativistic case, we have ()
Since the only nonvanishing connection coefficient is, then we can find the acceleration of the test particle:
Thus, the gravitational force acting on the particle in cavity is linear in its radius, quadric in mass of graviton and directs outward the center, with factor being positive. All the particles are attracted by the shell and tend to “fall” on it.
The interesting coincidence is that the expressions for the strength of electromagnetic field to act on the test charge and that of weak gravitational field to act on test mass are the same values, but having opposite signs:
The naive attempt to find test mass acceleration inside the shell in the frameworks of Newtonian approach gives the correct result for some choice of potential. In coordinates, where the unperturbed metrics is Galilean, one can use for such gravitational potential an expression
which yields the correct strength of the gravitational field (acceleration),coinciding with Equation (30):
If one use the minimal upper limit for the mass of graviton, where is the Hubble constant, then the order of magnitude of this acceleration is times less than freefall acceleration. For the laboratory size cavity, this is far from experimental opportunities since. Besides, any tiny distortion of the shell surface will produce the additional gravitation force, competing with m2- force (30) and worsening the prospects.
At this end, let’s calculate the energy density of the gravitational field in cavity. In linear approximation, an energy density (00-component of symmetric Hilbert stress-energy tensor, generalizing the Landau-Lifshitz com-
plex on the massive case) is constant and negative, having the order of magnitude:
It differs from the energy in electromagnetic case (8) by the coefficient―7.
Contrary to the Pauli-Firtz massive gravitation  , the solutions (27)-(29) and the stress-energy tensor for finite-range theory  -  have the correct limit.
4. Breakdown of Gravitational Screening―Local Hubble Flow in the Nearby Universe
If the effect (30) is too small for the lab sizes, can it be pertinent in cosmos, when the distance W is big enough?
Indeed, in the cavity the local Hubble low takes place
where the “Hubble constant” equals to
This result suggests to search this effect for the group of the most massive cosmic objects, which nevertheless can be considered as the pointlike ones, moving in the mutual gravitational field, provided the dark matter should not preclude such motion, cause it (dm) should be concentrated inside these bodies. Galaxy stars don’t fit due to the distributed dark matter and not big enough distances W. Then the best option is the Local Group of Galaxies, which consists of two massive galaxies―Milky Way and M31 Galaxy and about 50 more light galaxies  . All these objects locate in nearby Universe at the redshifts and can be considered as the point like test masses. Other such systems are located much far from us and less searched.
How to apply the result (30) to the Local Group of Galaxies? We can visualize the sphere containing all these galaxies with the origin in the center of mass. Outer gravitational field is the cosmological one, which is enormous at the present time  :
where a―FLRW-scale factor, and consequently one can neglect the Newtonian field of the very galaxies. Therefore, we have strong inequality. The dark energy, which we didn’t yet take into account, is uniformly distributed inside this sphere. But it is their homogeneity that enables us, contrary to the dark matter, to calculate its contribution to the magnitude of the effect.
Qualitatively one can do this when notice that the dark energy (the quintessence in our case   ) with relative density, and satisfying the equation of state
enters the Friedmannian cosmological gravitational field equations additively with m2-term  :
where―critical density. Since a) the relative CMB density at the present time is very small, b) the dark matter inside our sphere is concentrated in galaxies and missed in inter galaxies vacuum, and taking into account c), for the quantitative evaluation of our effect we can substitute
Therefore from (30), (31), we can get the equation of motion for any galaxy in Local Group, taking into account the Newtonian term, where M is the mass of Local Group, W? distance from its center of mass:
This equation describes the local Hubble flow of galaxies with bigger velocities of more distant galaxies. If the distances W are small enough, then attractive Newtonian term predominates the repulsive dark energy. The
second Hubble term prevails at the distances. It’s clearly distinguished on the velocities-distances diagram of Local Group () and Local Flow () from the Karachent-
sev et al. paper  , based on HST data (Figure 1). Points represent galaxies with measured radial velocities and distances, calculated from the Group center of mass. It, in their turn, has 600 km/s speed with respect to the CMB  .
As it follows from the diagram, all the galaxies have been separated into two parts―the inner Local Group and external Local Flow. The flow galaxies have only positive velocities―they recede from the Local Group, where the motion of bodies (galaxies) has no definite direction and they move with different speeds (positive and negative).
Figure 1. Velocity-distance diagram for galaxies at distances up to 3 Mpc. Each dot corresponds to a galaxy with measured distance and radial velocity in the reference frame associated with the center of mass of the Local group. The velocities are deemed positive if they are directed away from the group center.
Let’s point out, that such simple spherically-symmetric model, where The Local Group is represented by the mass M, and the galaxies-by the pointlike bodies with the masses much less than M, on the backgroung of dark energy with constant density given by cosmological constant, first considered by Chernin, Teericorpi and Baryshev  -  in the frameworks of General Relavity.
Rigorous calculation of the model in relativistic theory of gravitation  with quintessence as the dark energy(cosmological term in the theory  is ruled out by the causality principle)have been performed in  , where the final result
is very close to (32) and differs only by the factor at the. As it follows from (32) and (33), the mass of the graviton weakens the repulsive force of dark energy and plays the role of the negative cosmological constant.
Comparing (33) with the results of observations  and using independent evaluation of, the quintessence parameter was strongly constrained in RTG  :
so the factor is very close to 1:
Strong limits of quintessence parameter in RTG (34) are very close to the last range of dark energy equation of state in LCDM model, established from combined data including Planck satellite, Type Ia supernovae, etc.  :
Authors thank A. A. Logunov, M. A. Mestvirishvili and Yu. V. Baryshev for useful discussions and A. V. Selikhov for support.