Phase Diagram of an S = 1/2 J1-J2 Anisotropic Heisenberg Antiferromagnet on a Triangular Lattice

Show more

1. Introduction

Over the past three decades, the low temperature properties of low-dimensional quantum systems have been studied because of the exotic spin states that can arise from quantum fluctuations. The quantum antiferromagnetic Heisenberg (QAFH) model on a triangular lattice is a typical quantum frustrated system. This involves a generalized model with an antiferromagnetic nearest-neighbor (NN) interaction ${J}_{1}\left(>0\right)$ and a next-nearest-neighbor (NNN) interaction ${J}_{2}$ , the model Hamiltonian of which given by

$H=2{J}_{1}{\displaystyle \underset{\langle i\mathrm{,}j\rangle}{\sum}}\left[{S}_{i}^{x}{S}_{j}^{x}+{S}_{i}^{y}{S}_{j}^{y}+\alpha {S}_{i}^{z}{S}_{j}^{z}\right]+2{J}_{2}{\displaystyle \underset{\langle \langle i\mathrm{,}j\rangle \rangle}{\sum}}\left[{S}_{i}^{x}{S}_{j}^{x}+{S}_{i}^{y}{S}_{j}^{y}+\alpha {S}_{i}^{z}{S}_{j}^{z}\right]\mathrm{,}$ (1)

where
${S}_{i}^{\nu}$ is the
$\nu $ component (
$\nu =x,y,z$ ) of the quantum spin
$S=1/2$ at lattice site i,
$\alpha \left(\equiv {J}_{z}/{J}_{xy}\right)$ is an exchange anisotropy, and the sums
$\langle i\mathrm{,}j\rangle $ and
$\langle \langle i\mathrm{,}j\rangle \rangle $ run over all NN and NNN pairs of sites, respectively. Hereafter we set
${J}_{1}=1$ as the unit of energy scaling. The ground state (GS) of an isotropic QAFH model (
$\alpha =1$ ) with
${J}_{2}=0$ is the central issue of the model. Anderson proposed a resonating-valence-bond state or a spin liquid (SL) state as the GS of the model [1] . Since then, many authors have used various methods to study the model [2] - [16] . The GS of the model is now widely believed to have the classical long-range-order (LRO) in the form of the 3-sublattice structure (120˚ Néel state). However recent experiments on model compounds such as κ-(ET)_{2}Cu_{2}(CN)_{3} [17] , EtMe_{3}Sb[Pd(dmit)_{2}]_{2} [18] , and Ba_{3}IrTi_{2}O_{9} [19] have observed no LRO down to very low temperatures. Motivated by this discrepancy, the present authors [14] (hereafter referred to as SMFS) reexamined the GS of an anisotropic QAFH model with
$0\le \alpha <\infty $ on finite lattices having used an exact diagonalization technique, and found that the classical LRO is absent for
$0.55\lesssim \alpha \lesssim 1.67$ . This includes the GS of the isotropic model (
$\alpha =1$ ) with
${J}_{2}=0$ , i.e., it has no LRO and is the SL state.

In the present paper, we consider the effect of the NNN interaction
${J}_{2}$ on the anisotropic QAFH model. The isotropic QAFH model with
${J}_{2}$ was studied recently using approximations such as a variational Monte Carlo (VMC) method [20] , a many-variable VMC (mVMC) method [21] , a coupled cluster method (CCM) [22] , and a density matrix renormalization group method [23] [24] . These approaches showed that the 120˚ Néel state occurs for
${J}_{2}<{J}_{2}^{\left(t\right)}$ and that a four-sublattice antiferromagnetic LRO state (stripe Néel state) occurs for
${J}_{2}>{J}_{2}^{\left(s\right)}$ , i.e., the SL state appears between them,
${J}_{2}^{\left(t\right)}\lesssim {J}_{2}\lesssim {J}_{2}^{\left(s\right)}$ . The thresholds have been estimated as
${J}_{2}^{\left(t\right)}\sim 0.06$ and
${J}_{2}^{\left(s\right)}\sim 0.16$ [20] [22] [23] [24] , although the mVMC method suggested a smaller region of the SL state, i.e.,
${J}_{2}^{\left(t\right)}\simeq 0.10$ and
${J}_{2}^{\left(s\right)}\simeq 0.135$ [21] . However, an exact result for finite lattices by SMFS [14] suggested
${J}_{2}^{\left(t\right)}<0$ . No studies have been carried out on
${J}_{2}^{\left(t\right)}$ and
${J}_{2}^{\left(s\right)}$ using the exact diagonalization technique. We therefore apply the same method used by SMFS to the model extended with
${J}_{2}$ . We calculate exactly the squared sublattice magnetization of finite lattices, and we estimate
${J}_{2}^{\left(t\right)}$ and
${J}_{2}^{\left(s\right)}$ using their Binder ratios over a wide range of
$\alpha $ and try to draw the phase diagram in theα-J_{2} plane.

In Section 2, we present our method with the finite lattices. In Section 3, we estimate the threshold ${J}_{2}^{\left(t\right)}\left(\alpha \right)$ between the 120˚ Néel state and the SL state. In Section 4, we consider the stripe Néel state and its threshold ${J}_{2}^{\left(s\right)}\left(\alpha \right)$ with the SL state. In Section 5, we propose a phase diagram of the model.

2. Method

It is known that the GS of the classical model is a 120˚ Néel state when ${J}_{2}<{J}_{2}^{\left(clas\right)}$ , and the stripe Néel state when ${J}_{2}>{J}_{2}^{\left(clas\right)}$ , where ${J}_{2}^{\left(clas\right)}=1/8$ for $\alpha \le 1$ and tends to zero as $\alpha \to \infty $ . The unit cells of these states are shown in Figure 1. Although a similar LRO may be expected to appear in the anisotropic QAFH model, the classical ordered state is not a good quantum state. Therefore a remarkable difference may exist in the phase transition between the classical and quantum models.

We first consider this problem. Hereafter we refer to the spin space of a lattice with $N=3n$ sites with three-sublattice symmetry as the three-sublattice space (3SLS), where n is a natural number. Similarly, we refer to the spin space of a lattice with $N=4n$ sites with four-sublattice symmetry as the four-sublattice space (4SLS). The minimum energies per site in the 3SLS and 4SLS are labeled as ${E}_{tri}$ and ${E}_{str}$ , respectively. The spin state is in the 3SLS when ${E}_{tri}<{E}_{str}$ and in the 4SLS when ${E}_{str}<{E}_{tri}$ . In the classical model, the threshold of ${J}_{2}^{\left(clas\right)}$ is one at which the spin space changes from one to the other, and the phase transition at ${J}_{2}^{\left(clas\right)}$ is of the first order. In the quantum model, although the spin space changes at some threshold ${J}_{2}^{\left(quan\right)}$ , no phase transition will take place at ${J}_{2}^{\left(quan\right)}$ , because there would be no LRO in those spin spaces at ${J}_{2}\sim {J}_{2}^{\left(quan\right)}$ . We must then consider the thresholds and natures of the phase transitions in the 3SLS and in the 4SLS, separately.

For the 3SLS, we consider the lattices with $N=18\text{\hspace{0.17em}}\text{-}\text{\hspace{0.17em}}30$ (and partly $N=36$ ) sites with periodic boundary conditions suitable for the three-sublattice structure (Figure 2(a)).

For the 4SLS, we consider the lattices with N = 24, 28, and 32 sites with periodic boundary conditions suitable for the stripe structure (Figure 2(b)).

Figure 1. (a) Sublattices A, B, and C in the three-sublattice state; (b) Sublattices A, B, C, and D in the four-sublattice state.

Figure 2. (a) The lattices with three-sublattice symmetry (3SLS). The lattices of N = 21, 27, and 36 appear in Ref. [10] ; (b) The lattices with four-sublattice symmetry (4SLS).

In either case, we obtain the GS eigenfunction ${|{\psi}_{sG}\rangle}_{N}$ of the N sites using the Lanczos method, where s = tri or str for the 3SLS or 4SLS, respectively. The $\nu $ component of the magnetization on the ${\Omega}_{l}$ sublattice is defined as

${\mu}_{l}^{\nu}=\frac{2{N}_{sub}}{N}{\displaystyle \underset{i\in {\Omega}_{l}}{\sum}}{S}_{i}^{\nu},$ (2)

where ${N}_{sub}=3$ and $l=A,B$ , and C for the 3SLS, and ${N}_{sub}=4$ and $l=A,B,C$ , and D for the 4SLS. The operators of the z, xy, and xyz components of the squared sublattice magnetization are defined as

${m}_{2}^{z}=\frac{1}{{N}_{sub}}{\displaystyle \underset{l}{\sum}}{\left({\mu}_{l}^{z}\right)}^{2},$ (3)

${m}_{2}^{xy}=\frac{1}{{N}_{sub}}{\displaystyle \underset{l}{\sum}}\left({\left({\mu}_{l}^{x}\right)}^{2}+{\left({\mu}_{l}^{y}\right)}^{2}\right),$ (4)

${m}_{2}^{xyz}=\frac{1}{{N}_{sub}}{\displaystyle \underset{l}{\sum}}\left({\left({\mu}_{l}^{x}\right)}^{2}+{\left({\mu}_{l}^{y}\right)}^{2}+{\left({\mu}_{l}^{z}\right)}^{2}\right).$ (5)

We calculate the $\zeta $ component the squared sublattice magnetization, ${\langle {m}_{\mathrm{2,}s}^{\zeta}\rangle}_{N}$ , as

${\langle {m}_{\mathrm{2,}s}^{\zeta}\rangle}_{N}={\mathrm{}}_{N}\langle {\psi}_{sG}|{m}_{2}^{\zeta}{|{\psi}_{sG}\rangle}_{N}$ (6)

where $\zeta =z$ , xy, or xyz.

We study the Binder ratios [25] that are used by SMFS [14] to estimate the threshold $\alpha $ of the model with ${J}_{2}=0$ . At the critical point, the Binder ratio is size invariant. If there is a LRO, the Binder ratio is expected to increase with the system size. In contrast, in the paramagnetic or SL state, the Binder ratio decreases with the system size. This means that the size dependence of the Binder ratio is different from each other with and without a LRO. The z, xy, and xyz components of the Binder ratio can be defined as

${B}_{s}^{z}\left(N\right)=\left(3-{\langle {\left({m}_{\mathrm{2,}s}^{z}\right)}^{2}\rangle}_{N}/{\langle {m}_{\mathrm{2,}s}^{z}\rangle}_{N}^{2}\right)/2\mathrm{,}$ (7)

${B}_{s}^{xy}\left(N\right)=2-{\langle {\left({m}_{2,s}^{xy}\right)}^{2}\rangle}_{N}/{\langle {m}_{2,s}^{xy}\rangle}_{N}^{2},$ (8)

${B}_{s}^{xyz}\left(N\right)=\left(5-3{\langle {\left({m}_{\mathrm{2,}s}^{xyz}\right)}^{2}\rangle}_{N}/{\langle {m}_{\mathrm{2,}s}^{xyz}\rangle}_{N}^{2}\right)/2\mathrm{.}$ (9)

Before estimating ${J}_{2}^{\left(t\right)}$ and ${J}_{2}^{\left(s\right)}$ , we should examine that no phase transition will take place at ${J}_{2}^{\left(quan\right)}$ . Figure 3 shows ${E}_{tri}$ and ${E}_{str}$ together with ${\langle {m}_{\mathrm{2,}tri}^{xy}\rangle}_{N}$ and ${\langle {m}_{\mathrm{2,}str}^{xy}\rangle}_{N}$ for the case of $\alpha =0.4$ . As mentioned above, the spin space changes at ${J}_{2}={J}_{2}^{\left(quan\right)}\left(\sim 0.065\right)$ , whereas no signal of a change in the magnetic state is seen at this point. We consider the 3SLS for ${J}_{2}<{J}_{2}^{\left(quan\right)}$ and the 4SLS for ${J}_{2}>{J}_{2}^{\left(quan\right)}$ . A remarkable point is that ${\langle {m}_{\mathrm{2,}tri}^{xy}\rangle}_{N}$ ( ${\langle {m}_{\mathrm{2,}str}^{xy}\rangle}_{N}$ ) changes markedly around ${J}_{2}^{peak}$ , at which ${E}_{tri}$ ( ${E}_{str}$ ) has its maximum value. In the 3SLS, a bending of ${E}_{tri}$ accompanied by a discontinuous drop of ${\langle {m}_{\mathrm{2,}tri}^{xy}\rangle}_{N}$ indicates exchange in the GS between the lowest and next-lowest energy eigenstates at ${J}_{2}^{peak}$ . However, we reason that this says nothing about the

Figure 3. The GS energies ${E}_{s}$ and the squared sublattice magnetizations ${\langle {m}_{\mathrm{2,}s}^{xy}\rangle}_{N}$ of the 3SLS ( $s=tri$ ) and the 4SLS ( $s=str$ ). The data of the 4SLS for $N=30$ are averages of those of $N=28$ and $N=32$ . The solid and open symbols are ${E}_{s}$ and ${\langle {m}_{\mathrm{2,}s}^{xy}\rangle}_{N}$ , respectively. An arrow represents the positions of ${J}_{2}^{\left(quan\right)}$ .

phase transition between the 120˚ Néel state and the SL state because ${J}_{2}^{peak}>{J}_{2}^{\left(quan\right)}$ . The phase boundary should be estimated by a different method. In contrast, we expect ${J}_{2}^{\left(s\right)}$ to be near ${J}_{2}^{peak}$ , because ${J}_{2}^{\left(quan\right)}<{J}_{2}^{peak}$ . In Section 4, we consider ${J}_{2}^{peak}$ together with the Binder ratio ${B}_{str}^{\zeta}$ in order to estimate ${J}_{2}^{\left(s\right)}$ .

3. Three-Sublattice Néel State

In this section, we estimate the threshold ${J}_{2}^{\left(t\right)}$ . We consider ${\langle {m}_{\mathrm{2,}tri}^{\zeta}\rangle}_{N}$ in the GS of the 3SLS. Special attention should be paid to the ${M}^{z}\left(={\displaystyle {\sum}_{i}}\text{\hspace{0.05em}}{S}_{i}^{z}\right)$ subspace in which the GS belongs. For $\alpha \le 1$ , the GS is in the minimum $\left|{M}^{z}\right|$ subspace. For $\alpha >1$ , however, the GS may not be restricted to the minimum $\left|{M}^{z}\right|$ subspace depending on ${J}_{2}$ . We then consider the cases $\alpha \le 1$ and $\alpha >1$ separately.

3.1. XY-Like Case (α ≤ 1)

For $\alpha \le 1$ , the LRO has the 120˚ Néel state symmetry, and ${\langle {m}_{2,tri}^{xy}\rangle}_{N}$ and ${\langle {m}_{2,tri}^{xyz}\rangle}_{N}$ are calculated for various $\alpha $ . For $\alpha <1$ , because the spins lie in the xy plane, we consider the $\zeta =xy$ component, whereas the $\zeta =xyz$ component for $\alpha =1$ . In Figures 4(a)-(d), we present ${\langle {m}_{\mathrm{2,}tri}^{\zeta}\rangle}_{N}$ for $\alpha =0.0$ , 0.4, 0.8, and 1.0 as functions of ${J}_{2}$ , respectively. As ${J}_{2}$ is decreased, ${\langle {m}_{\mathrm{2,}tri}^{\zeta}\rangle}_{N}$ increases revealing the development of the 120˚ spin correlation. For small $\alpha \left(=0,0.4\right)$ , the finite-size effect (FSE) for ${J}_{2}\lesssim 0.05$ is rather weak implying the occurrence of the 120˚ Néel state. As $\alpha $ is increased, the FSE becomes stronger. In the isotropic case of $\alpha =1.0$ , we can see a strong FSE even for ${J}_{2}<-\mathrm{0.1.}$

Figure 4. The squared three-sublattice magnetizations (a)-(c) ${\langle {m}_{\mathrm{2,}tri}^{xy}\rangle}_{N}$ and (d) ${\langle {m}_{\mathrm{2,}tri}^{xyz}\rangle}_{N}$ as functions of ${J}_{2}$ . Note that for $\alpha =1.0$ , ${\langle {m}_{2,tri}^{xyz}\rangle}_{N}=\left(3/2\right){\langle {m}_{2,tri}^{xy}\rangle}_{N}$ .

We consider the Binder ratio ${B}_{tri}^{\zeta}\left(N\right)$ [25] [14] . In Figures 5(a)-(d), we show ${B}_{tri}^{\zeta}\left(N\right)$ for different even N as functions of ${J}_{2}$ for $\alpha =0$ , 0.4, 0.8, and 1.0, respectively. For a large ${J}_{2}$ , ${B}_{tri}^{\zeta}\left(N\right)$ decreases with increasing N, which reveals that LRO is absent. As ${J}_{2}$ is decreased, ${B}_{tri}^{\zeta}\left(N\right)$ increases and the values for different N are close to each other. For $\alpha \sim 0$ , ${B}_{tri}^{\zeta}\left(N\right)$ for different N intersect at almost the same ${J}_{2}$ (see Figure 5(a) and Figure 5(b)). That is, ${J}_{2}^{\left(t\right)}\left(0\right)=0.06\pm 0.01$ and ${J}_{2}^{\left(t\right)}\left(0.4\right)=0.01\pm 0.01$ . As $\alpha $ is increased, the intersection points scatter, as seen in Figure 5(c). In this case, we consider a lower bound ${J}_{2}^{l}$ and a upper bound ${J}_{2}^{u}$ of ${J}_{2}^{\left(t\right)}\left(\alpha \right)$ according to the hypotheses described in Sec. II: the LRO is present when the Binder ratio ${B}_{s}^{\zeta}\left(N\right)$ increases with N, whereas it is absent when ${B}_{s}^{\zeta}\left(N\right)$ decreases with increasing N. We evaluate ${J}_{2}^{l}$ and ${J}_{2}^{u}$ , and tentatively estimate ${J}_{2}^{\left(t\right)}\left(\alpha \right)$ as their average value. In this way, we get the thresholds as ${J}_{2}^{\left(t\right)}\left(0.6\right)=-0.01\pm 0.01$ , ${J}_{2}^{\left(t\right)}\left(0.8\right)=-0.05\pm 0.04$ , and ${J}_{2}^{\left(t\right)}\left(0.9\right)=-0.10\pm 0.06$ .

In the case of $\alpha =1.0$ , ${B}_{tri}^{\zeta}\left(N\right)$ exhibits a somewhat different behavior from those for $\alpha <1.0$ . Although ${B}_{tri}^{\zeta}\left(N\right)$ increases with decreasing ${J}_{2}$ , its increment

Figure 5. Binder ratios (a)-(c) ${B}_{tri}^{xy}\left(N\right)$ and (d) ${B}_{tri}^{xyz}\left(N\right)$ of the squared 3-sublattice magnetization as functions of ${J}_{2}$ .

depends only very weakly on N, especially for ${J}_{2}<0$ . We could see no definite intersection point of ${B}_{tri}^{\zeta}\left(N\right)$ for $N\ge 24$ down to ${J}_{2}=-0.4$ , i.e., we could not evaluate ${J}_{2}^{l}$ . Therefore we believe ${J}_{2}^{\left(t\right)}\left(1\right)<-0.1$ , because ${J}_{2}^{u}\sim -0.1$ .

3.2. Ising-Like Case (α > 1)

For $\alpha >1$ , we are interested in ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}$ and ${\langle {m}_{2,tri}^{xy}\rangle}_{N}$ because a distorted 120˚ Néel state occurs in the classical model. We obtain the eigenfunction ${|\psi \left({M}^{z}\right)\rangle}_{N}$ with the minimum energy $E\left({M}^{z}\right)$ for each ${M}^{z}$ subspaces. Note that we consider only the subspaces of ${M}^{z}\le N/6$ because the GS is in the ${M}^{z}=N/6$ subspace for ${J}_{2}\to -\infty $ . The GS eigenfunction ${|{\psi}_{sG}\rangle}_{N}$ of the system is one which gives the lowest value among $E\left({M}^{z}\right)$ ’s. When ${J}_{2}\ge 0$ , the GS eigenfunction ${|{\psi}_{sG}\rangle}_{N}$ is ${|\psi \left(0\right)\rangle}_{N}$ . As ${J}_{2}$ is decreased, ${|{\psi}_{sG}\rangle}_{N}$ successively changes to ${|\psi \left(1\right)\rangle}_{N},{|\psi \left(2\right)\rangle}_{N},\cdots ,{|\psi \left(N/6\right)\rangle}_{N}$ at ${J}_{2}^{\left(1\right)},{J}_{2}^{\left(2\right)},\cdots ,{J}_{2}^{\left(N/6\right)}$ , respectively. Using ${|{\psi}_{sG}\rangle}_{N}$ , we obtain ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}$ and ${\langle {m}_{2,tri}^{xy}\rangle}_{N}$ for various $\alpha $ . A typical result of these is shown in Figure 6 for $\alpha =1.25$ . For ${J}_{2}\sim 0$ , both ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}$ and ${\langle {m}_{2,tri}^{xy}\rangle}_{N}$ exhibit a strong N dependence which reveals that ${\langle {m}_{\mathrm{2,}tri}^{xy}\rangle}_{N}\mathrm{,}{\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}\to 0$ for $N\to \infty $ . As ${J}_{2}$ is decreased, ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}$ and ${\langle {m}_{2,tri}^{xy}\rangle}_{N}$ show different behaviors from each other. For ${\langle {m}_{2,tri}^{xy}\rangle}_{N}$ , they remain almost constant down to ${J}_{2}^{\left(1\right)}$ and drop at ${J}_{2}^{\left(1\right)},{J}_{2}^{\left(2\right)},\cdots ,{J}_{2}^{\left(N/6\right)}$ . For the whole range of ${J}_{2}$ , we see a strong N dependence, that suggests that ${\langle {m}_{2,tri}^{xy}\rangle}_{N}\to 0$ for $N\to \infty $ . In contrast, ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}$ gradually increases down to ${J}_{2}^{\left(1\right)}$ and discontinuously jumps at ${J}_{2}^{\left(1\right)},{J}_{2}^{\left(2\right)},\cdots ,{J}_{2}^{\left(N/6\right)}$ . It exhibits its own N dependence in different ranges of ${J}_{2}$ . For 1) ${J}_{2}<{J}_{2}^{\left(N/6\right)}$ , ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}$ is almost independent of N. We believe that the classical ferrimagnetic state arises in this range, because ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}\sim 1$ and ${\langle {m}_{2,tri}^{xy}\rangle}_{\infty}=0$ . For 2) ${J}_{2}^{\left(N/6\right)}<{J}_{2}<{J}_{2}^{\left(1\right)}$ , ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}$ increases with N revealing the occurrence of the LRO of the z component of the spin. However, for 3) ${J}_{2}^{\left(1\right)}<{J}_{2}$ , ${\langle {m}_{\mathrm{2,}tri}^{z}\rangle}_{N}$ slightly decreases with increasing N. Note that we also obtain the similar results for $\alpha =1.67$ and 2.5.

In Figure 7(a) and Figure 7(b), we plot the Binder ratio ${B}_{tri}^{z}\left(N\right)$ as a function

Figure 6. Squared three-sublattice magnetizations ${\langle {m}_{2,tri}^{z}\rangle}_{N}$ (solid symbols) and ${\langle {m}_{2,tri}^{xy}\rangle}_{N}$ (open symbols) as functions of ${J}_{2}$ . Jumps in those quantities occur at ${J}_{2}^{\left(1\right)},{J}_{2}^{\left(2\right)},\cdots ,{J}_{2}^{\left(N/6\right)}$ from the right.

Figure 7. Binder ratios ${B}_{tri}^{z}\left(N\right)$ as a functions of ${J}_{2}$ for (a) $\alpha =1.25$ and (b) $\alpha =2.5$ .

of ${J}_{2}$ for $\alpha =1.25$ and $\alpha =2.5$ , respectively. For (a) $\alpha =1.25$ , we evaluate the lower and upper bounds of ${J}_{2}^{\left(t\right)}$ as ${J}_{2}^{l}\sim -0.15$ and ${J}_{2}^{u}\sim 0$ , i.e., ${J}_{2}^{\left(t\right)}\left(1.25\right)\sim -0.08\pm 0.08$ . For (b) $\alpha =2.5$ , we estimate ${J}_{2}^{l}\sim -0.08$ and ${J}_{2}^{u}\sim 0.04$ , i.e., ${J}_{2}^{\left(t\right)}\left(2.5\right)\sim -0.02\pm 0.06$ . Note that we also examined ${B}_{tri}^{xy}\left(N\right)$ to confirm the speculation given above and found that, in fact, ${B}_{tri}^{xy}\left(N\right)$ decreases with increasing N for the whole range of ${J}_{2}$ .

To close this subsection, we emphasize that the distorted 120˚ Néel state is absent in the QAFH model, in contrast to the classical model. We find that, when ${J}_{2}<{J}_{2}^{\left(1\right)}$ , the LRO of the z component of the spin occurs. A question remains as to what the value of ${J}_{2}^{\left(1\right)}$ for $N\to \infty $ , ${J}_{\mathrm{2,}\infty}^{\left(1\right)}$ . If ${J}_{2,\infty}^{\left(1\right)}={J}_{2}^{\left(t\right)}\left(\alpha \right)$ , the LRO of the z component of the spin occurs for ${J}_{2}\le {J}_{2}^{\left(t\right)}\left(\alpha \right)$ . If not, two possibilities exist in the range ${J}_{2,\infty}^{\left(1\right)}<{J}_{2}<{J}_{2}^{\left(t\right)}\left(\alpha \right)$ : either there is still the LRO, or the system is in a critical state that is similar to the spin state of the Ising model with ${J}_{2}=0$ . Further studies are necessary to answer this question.

4. Stripe State

In this section, we consider the stripe state. We obtain the GS as the eigenfunction ${|{\psi}_{strG}\rangle}_{N}={|\psi \left(0\right)\rangle}_{N}$ with energy ${E}_{str}$ because the stripe state belongs to the ${M}^{z}=0$ subspace. In Figure 8(a) and Figure 8(b), we show these quantities as functions of ${J}_{2}$ for $\alpha =0.4$ and $\alpha =1.25$ , respectively. We readily see that the results for the $N=24$ system are quantitatively different from those of the $N=28$ and 32 systems, which lead us to consider mainly data for $N\ge 28$ in order to evaluate ${J}_{2}^{\left(s\right)}$ . We see similar properties in the cases of $\alpha <1$ and $\alpha >1$ . The magnetization ${\langle {m}_{\mathrm{2,}str}^{\zeta}\rangle}_{N}$ rapidly increases around the peak posion ${J}_{2}^{peak}$ of ${E}_{str}$ that implies the occurrence of the phase transition. When ${J}_{2}<{J}_{2}^{peak}$ , ${\langle {m}_{\mathrm{2,}str}^{\zeta}\rangle}_{N}$ is small and its N dependence is strong which reveals that the stripe state is absent. When ${J}_{2}>{J}_{2}^{peak}$ , ${\langle {m}_{\mathrm{2,}str}^{\zeta}\rangle}_{N}$ is large, although its N dependence is still considerable especially for $\alpha <1$ . Then we examine the

Figure 8. The GS energies ${E}_{str}$ and four-sublattice magnetizations ${\langle {m}_{tri}^{\zeta}\rangle}_{N}$ as functions of ${J}_{2}$ .

Figure 9. Binder ratios as functions of ${J}_{2}$ .

Binder ratio ${B}_{str}^{\zeta}\left(N\right)$ . In Figure 9(a) and Figure 9(b), we plot ${B}_{str}^{xy}\left(N\right)$ and ${B}_{str}^{z}\left(N\right)$ as functions of ${J}_{2}$ for $\alpha =0.4$ and $\alpha =1.25$ , respectively. For $\alpha =0.4$ , when ${J}_{2}\gtrsim 0.19$ , ${B}_{str}^{xy}\left(N\right)$ increases with N, suggesting the presence of the stripe state, i.e., ${J}_{2}^{u}=0.19$ . In contrast, the lower bound of ${J}_{2}^{\left(s\right)}$ is evaluated from ${J}_{2}^{peak}\sim 0.16$ of the $N=32$ system because ${J}_{2}^{peak}$ increases slightly with N. Therefore we estimate ${J}_{2}^{\left(s\right)}\left(0.4\right)=0.18\pm 0.02$ . Similarly, we estimate ${J}_{2}^{\left(s\right)}\left(1.25\right)=0.195\pm 0.010$ .

We have also examined ${J}_{2}^{\left(s\right)}$ for $\alpha =1$ . We may evaluate the lower bound of ${J}_{2}^{l}={J}_{2}^{peak}\sim 0.20$ . However, we could not evaluate ${J}_{2}^{u}$ because ${B}_{str}^{xyz}\left(32\right)<{B}_{str}^{xyz}\left(28\right)$ even up to ${J}_{2}=0.3$ [26] .

5. Summary

We have studied the
$S=1/2$ anisotropic antiferromagnetic model
$\left(\alpha \equiv {J}^{z}/{J}^{xy}\right)$ with nearest-neighbor (J_{1}) and next-nearest-neighbor (J_{2}) interactions on a triangular lattice using the exact diagonalization method. We have obtained the ground-state energy and the sublattice magnetizations for systems of different size N. We have examined Binder ratios to investigate the stability of the long-range order of the system. The N-dependences of Binder ratios suggest the threshold
${J}_{2}^{\left(t\right)}\left(\alpha \right)$ between the three-sublattice Néel state and the disordered state, i.e., the spin liquid (SL) state, and the threshold
${J}_{2}^{\left(s\right)}\left(\alpha \right)$ between the stripe state and the SL state. The results are summarized in the phase diagram shown in Figure 10. For
$\alpha <1$ , the classical 120˚ phase or the stripe phase occurs in the xy plane. For
$\alpha >1$ , the xy component of the sublattice magnetization vanishes, i.e., the distorted 120˚ state is replaced by the collinear (up up down) antiferromagnetic state because of quantum fluctuations.

We have suggested that the SL state exists over a wide range in theα-J_{2} plane in contrast with recent approximation studies [20] [22] [23] [24] which give
${J}_{2}^{\left(t\right)}\left(1\right)=0.06\text{-}0.10$ and
${J}_{2}^{\left(s\right)}=0.135\text{-}0.16$ . The discrepancy will come from either the finite-size effect or approximations. Further studies are necessary to

Figure 10. The α-J_{2} phase diagram of the J_{1}-J_{2} anisotropic Heisenberg model on a triangular lattice. Cross symbols are those estimated in SMFS [14] .

establish the thresholds for $\alpha \sim 1$ .

Acknowledgements

Some of the results in this research were obtained using the supercomputing resources at the Cyberscience Center of Tohoku University.

References

[1] Anderson, P.W. (1973) Materials Research Bulletin, 8, 153-160.

https://doi.org/10.1016/0025-5408(73)90167-0

[2] Miyake, S.J. (1992) Journal of the Physical Society of Japan, 61, 983-988.

https://doi.org/10.1143/JPSJ.61.983

[3] Huse, D.A. and Elser, V. (1998) Physical Review Letters, 60, 2351.

[4] Sindzingre, P., Lecheminant, P. and Lhuillier, C. (1994) Physical Review B, 50, 3108.

https://doi.org/10.1103/PhysRevB.50.3108

[5] Singh, R.R.P. and Huse, D.A. (1992) Physical Review Letters, 68, 1766.

https://doi.org/10.1103/PhysRevLett.68.1766

[6] Elstner, N., Singh, R.R.P. and Young, A.P. (1993) Physical Review Letters, 71, 1629.

https://doi.org/10.1103/PhysRevLett.71.1629

[7] Fujiki, S. and Betts, D.D. (1987) Canadian Journal of Physics, 65, 76-81.

https://doi.org/10.1139/p87-013

Fujiki, S. and Betts, D.D. (1987) Canadian Journal of Physics, 65, 489.

Fujiki, S. and Betts, D.D. (1986) Progress of Theoretical Physics, 87, 268.

[8] Nishimori, H. and Nakanishi, H. (1988) Journal of the Physical Society of Japan, 57, 262.

Nishimori, H. and Nakanishi, H. (1989) Journal of the Physical Society of Japan, 58, 2607.

https://doi.org/10.1143/JPSJ.58.2607

Nishimori, H. and Nakanishi, H. (1989) Journal of the Physical Society of Japan, 58, 3433-3433.

https://doi.org/10.1143/JPSJ.58.3433

[9] Bernu, B., Lhuillier, C. and Pierre, L. (1992) Physical Review Letters, 69, 2590.

Bernu, B., Lecheminant, P.., Lhuillier, C. and Pierre, L. (1994) Physical Review B, 50, 10048.

https://doi.org/10.1103/PhysRevB.50.10048

[10] Leung, P.W. and Runge, K.J. (1993) Physical Review B, 47, 5861.

https://doi.org/10.1103/PhysRevB.47.5861

[11] Capriotti, L., Trumper, A.E. and Sorella, S. (1999) Physical Review Letters, 82, 3899.

https://doi.org/10.1103/PhysRevLett.82.3899

[12] Richter, J., Schulenburg, J., Honecker, A. and Schmalfuss, D. (2004) Physical Review B, 70, Article ID: 174454.

https://doi.org/10.1103/PhysRevB.70.174454

[13] Rubin, P. and Sherman, A. (2005) Physics Letters A, 334, 312.

https://doi.org/10.1016/j.physleta.2004.11.028

[14] Suzuki, N., Matsubara, F., Fujiki, S. and Shirakura, T. (2014) Physical Review B, 90, Article ID: 184414.

https://doi.org/10.1103/PhysRevB.90.184414

[15] White, S.R. and Chernyshev, A.L. (2007) Physical Review Letters, 99, Article ID: 127004.

https://doi.org/10.1103/PhysRevLett.99.127004

[16] Kulagin, S.A., Prokof’ev, N., Starykh, O.A., Svistunov, B. and Varney, C.N. (2013) Physical Review Letters, 110, Article ID: 070601.

https://doi.org/10.1103/PhysRevLett.110.070601

[17] Shimizu, Y., Miyagawa, K., Kanoda, K., Maesato, M. and Saito, G. (2003) Physical Review Letters, 91, Article ID: 107001.

[18] Itou, T., Oyamada, A., Maegawa, S., Tamura, M. and Kato, R. (2008) Physical Review B, 77, Article ID: 104413.

https://doi.org/10.1103/PhysRevB.77.104413

[19] Dey, T., Mahajan, A.V., Khuntia, P., Baenitz, M., Koteswararao, B. and Chou, F.C. (2012) Physical Review B, 86, Article ID: 140405.

[20] Mishmash, R.V., Garrison, J.R., Bieri, S. and Xu, C. (2013) Physical Review Letters, 111, Article ID: 157203.

https://doi.org/10.1103/PhysRevLett.111.157203

[21] Kaneko, R., Morita, S. and Imada, M. (2014) Journal of the Physical Society of Japan, 83, Article ID: 093707.

https://doi.org/10.7566/JPSJ.83.093707

[22] Li, P.H.Y., Bishop, R.F. and Campbell, C.E. (2015) Physical Review B, 91, Article ID: 014426.

https://doi.org/10.1103/PhysRevB.91.014426

[23] Zhu, Z. and White, S.R. (2015) Physical Review B, 92, 041105R.

https://doi.org/10.1103/PhysRevB.92.041105

[24] Hu, W.-J., Gong, S.-S., Zhu, W. and Sheng, D.N. (2015) Physical Review B, 92, 140403R.

https://doi.org/10.1103/PhysRevB.92.140403

[25] Binder, K. (1982) Zeitschrift für Physik B, 48, 319.

https://doi.org/10.1007/BF01305191

[26] Numerical Data Are.

http://web.tbgu.ac.jp/ait/nobu/trij1j2data.pdf