In this paper, the following fractional Klein-Gordon-Maxwell system is considered
where , denotes the fractional Laplacian operator, V is zero mass potential and K is a smooth function. When , system (1.1) reduces to a fractional Schrödinger equation. The fractional Schrödinger equation was first proposed by Laskin   as a result of expanding the Feynman path integral from the Brownian-like to the Lévy-like quantum mechanical paths. This kind of problem can apply to various fields. For example, Li et al.  studied a class of fractional Schrödinger equation with potential vanishing at infinity by using variational methods and obtained a positive solution for this equation. For more results about fractional Schrödinger equation, please see    and the references therein.
If , and , system (1.1) reduces to a Klein-Gordon-Maxwell equation, which was first studied by Benci and Fortunato  as a model describing a nonlinear Klein-Gordon equation interacting with an electromagnetic field with . For more details on the physical aspects of this problem, we refer the readers to see  and references therein.
When and , D’Aprile and Mugnai  investigated the following system
they obtained some results which complete the results obtained in .
In recent years, under various hypotheses on the potential and the nonlinearity , the existence of positive, multiple, ground state solutions for Klein-Gordon-Maxwell systems or similar systems, has been widely studied in the literature. For example, Azzollini and Pomponio  first proved the existence of a ground state solution for system (1.2) when the nonlinearity is more general. He  first considered a Klein-Gordon-Maxwell system with non-constant potential. Li and Tang  improved the result of . A nonlinear Klein-Gordon-Maxwell system with sign-changing potential was first considered by Ding and Li in . They obtained infinitely many solutions by symmetric mountain pass theorem. Otherwise, there are many works about the nonhomogeneous Klein-Gordon-Maxwell system. Wang  proved that a nonhomogeneous Klein-Gordon-Maxwell system had two solutions. In , Gan et al. obtained two solutions for a type of nonhomogeneous Klein-Gordon-Maxwell system with sign-changing potential. Another example is , Miyagaki et al. investigated system (1.1) with fractional Laplacian and f satisfied the following type of Ambrosetti-Rabinowitz condition:
(H4’) For all , There exists such that , where .
Inspired mainly by the aforementioned results, we find a ground state solution for (1.1) with potential vanishing at infinity. To show our result, we make the following assumptions first:
(H1) , and
or for any , there exists , where , such that
(H2) and . If (1.3) holds, then
If (1.4) holds, then
(H3) If (1.3) holds, then
If condition (1.4) holds, we assume that
(H4) There exists , such that for all .
To the best of our knowledge, Ambrosetti-Rabinowitz condition (AR condition for short) plays an important role in proving the boundedness of Palais-Smale sequence (PS sequence for short). In recent years, there are many papers devoted to replacing (AR) condition with weaker condition. It is easy to see that (H4) is weaker than (H4’). In this paper, we obtain a (PS) sequence by using the weaker (AR) condition. Besides, it seems that there is only one work about the Klein-Gordon-Maxwell system involving fractional Laplacian.
Theorem 1.1. Assume that and (H1)-(H4) hold. Then problem (1.1) admits a positive solution in E, where E is defined in Section 2.
In this paper, the main difficulty is lack of compactness of Sobolev embedding in whole space because of the nonlocal term and the fractional operator. To overcome this problem, we use the reduction method introduced by Caffarelli and Silvestre  and recover the compactness by the interaction of the behaviour of the potential and nonlinearity.
This paper is organized as follows. In Section 2, some preliminary results are presented. In Section 3, we give the proof of main result.
In this section, by the local reduction derived from Caffarelli and Silvestre , we first reformulate the nonlocal fractional system (1.1) into a local system, that is
where denotes the divergence of and such that
where , , and
is the outward normal derivative of . Similar definition is given for .
For and , the fractional Laplacian of is defined by
where denotes the Fourier transform, that is
where j denotes the imaginary unit. When is smooth enough, the of can be obtained by the following singular integral
where is a normalization constant and is the principle value.
For any , and are the completion of and , and endowed with the norms
respectively. The Sobolev space is defined by
which is the completion of under the norm
Let E be defined by
which is endowed with norm
then E is a Hilbert space. In the following, for convenience, for any u, let .
The functional associated to (2.1) is given by
which is of by (H1)-(H3).
A vector is a weak solution of system (2.1) if for any , i.e.
Lemma 2.1.  For every , there exists a unique which solves
Furthermore, in the set , we have for .
Let the weighted Banach space be
under the norm
The following Proposition 2.2 comes from the arguments in .
Proposition 2.2.  Assume that (H1) holds. Then
1) is compact for all , provided that (1.3) holds;
2) is compact provided that (1.4) holds;
3) If in E, then up to a subsequence
4) If in E, then up to a subsequence
5) If in E, then up to a subsequence, for any ,
Lemma 2.3.  If in E, as , then passing to a subsequence if necessary, weakly in , as .
Lemma 2.4. Assume that (H2) and (H3) hold. Then the functional satisfies
1) There exists such that if ;
2) There exists with such that .
The proof of Lemma 2.4 is standard, so we omit the details here.
From Lemma 2.4, there exists a sequence such that
3. Proof of Main Result
Lemma 3.1. Assume that (H2)-(H4) hold. Then the sequence given in (2.6) is bounded.
Proof. Let be a sequence of . Arguing indirectly, suppose such that
after passing to a subsequence. Denote . Then , in E and for a.e. . If , by the fact in , (2.2), (2.3), (2.4), (3.1) and Lemma 2.1, there are two cases to consider.
Case (1): . From (2.2), (2.3), (2.4) and (3.1), we derive
then , which contradict .
Case (2): . In this case, by (2.2), (2.3), (2.4), (3.1) and Lemma 2.1, one gets
then , which contradict .
If , then , where . For , we have as , and then, from (H4), we get
From (3.2) and Fatou’s Lemma, we obtain
From (2.2), (2.3), (3.1), (3.3) and Lemma 2.1, we have
a contradiction. Hence, the boundedness of in E is obtained.
Proof of Theorem 1.1. Let be a sequence given in (2.6). It follows from Lemma 3.1 that is bounded, passing to a subsequence, one can assume that there is such that
It suffices to show that , as . By Proposition 2.2, one has
From (2.4), we have
By , one gets
By Proposition 2.2, one obtains
From the proof of Lemma 2.3 in , we know that there exists a such that
and . Hence, from Lemma 2.3, we obtain that
Otherwise, since , one has
Hence, from (3.7) and (3.8), we have
which shows that
Hence, we conclude that
Thus, u is a ground state solution for . It follows from that . Since there is a sequence , we can obtain that u is positive from Lemma 2.1 by contradiction.
In this paper, we first reformulated the system (1.1) into a local system by using the local reduction. Then, we take advantage of the interaction of the behaviour of the potential and nonlinearity to recover the compactness. Meanwhile, we obtained a Palais-Smale sequence by using a weaker Ambrosetti-Rabinowitz condition. Finally, the existence of positive solution is proved by the mountain pass theorem. Obviously, the weaker Ambrosetti-Rabinowitz condition has been successfully applied to find the solution of the fractional Klein-Gordon-Maxwell system with potential vanishing at infinity. We hope that this result can be widely used in other systems.
The authors would like to thank the referees for their useful suggestions which have significantly improved the paper.
This work is supported by the National Natural Science Foundation of China (No. 11961014, No. 61563013) and Guangxi Natural Science Foundation (2016 GXNSFAA380082, 2018GXNSFAA281021).
 Li, Q.Q., Teng, K.M. and Wu, X. (2017) Existence of Positive Solutions for a Class of Critical Fractional Schrödinger Equations with Potential Vanishing at Infinity. Mediterranean Journal of Mathematics, 14, 80.
 Zhang, X., Zhang, B.L. and Repov š, D. (2016) Existence and Symmetry of Solutions for Critical Fractional Schrödinger Equations with Bounded Potentials. Nonlinear Analysis, 142, 48-68.
 Benci, V. and Fortunato, D. (2002) Solitary Waves of the Nonlinear Klein-Gordon Equation Coupled with the Maxwell Equations. Reviews in Mathematical Physics, 14, 409-420.
 D’Aprile, T. and Mugnai, D. (2004) Solitary Waves of the Nonlinear Klein-Gordon-Maxwell and Schrödinger-Maxwell Equations. Proceedings of the Royal Society of Edinburgh Section A, 134, 893-906.
 Ding, L. and Li, L. (2014) Infinitely Many Standing Wave Solutions for the Nonlinear Klein-Gordon-Maxwell System with Sign-Changing Potential. Computers & Mathematics with Applications, 68, 589-595.
 Wang, L.X. (2019) Two Solutions for a Nonhomogeneous Klein-Gordon-Maxwell System. Electronic Journal of Qualitative Theory of Differential Equations, 40, 1-12.
 Gan, C.L., Xiao, T. and Zhang, Q.F. (2020) Improved Results of Nontrivial Solutions for a Nonlinear Nonhomogeneous Klein-Gordon-Maxwell System Involving Sign-Changing Potential. Advances in Difference Equations, 167, 1-16.
 Miyagaki, O.H., de Moura, E.L. and Ruviaro, R. (2019) Positive Ground State Solutions for Quasicritical the Fractional Klein-Gordon-Maxwell System with Potential Vanishing at Infinity. Complex Variables and Elliptic Equations, 64, 315-329.
 Caffarelli, L. and Silvestre, L. (2007) An Extension Problems Related to the Fractional Laplacian. Communications in Partial Differential Equations, 32, 1245-1260.
 Alves, C.O. and Souto, M.A.S. (2013) Existence of Solutions for a Class of Nonlinear Schrödinger Equations with Potential Vanishing at Infinity. Journal of Differential Equations, 254, 1977-1991.