Novel Geometrical Frustration Effects in the Two-Dimensional Triangular-Lattice Antiferromagnet NiGa2S4 and Related Compounds

Geometrical frustration may suppress conventional magnetic long-range order and possibly promote a novel type of ordering associated with a higher order degree of freedom than spin dipolar-moment such as vector spin chirality and spin quadrupole moment. After a brief overview of such phenomena due to vector spin chirality, we focus on the two-dimensional (2D) frustrated magnetism in the layered chalcogenide Mott insulator NiGa2S4 and related compounds. NiGa2S4 provides the unique example of a $S = 1$ 2D antiferromagnet on a regular exact triangular lattice. Extensive studies using high-purity samples of NiGa2S4 have revealed that Ni2+ $S = 1$ Heisenberg spins exhibit resonant critical slowing down at $T^{*} = 8.5$ K without forming a magnetic long-range order, signaling a viscous spin-liquid state. The critical spin-fluctuation regime extends over almost an order of magnitude in temperature both above and below $T^{*}$. Even well below $T^{*}$, the spin–spin correlation remains short-ranged at an incommensurate wave vector close to $(1/6, 1/6, 0)$, corresponding to a 120° correlation with $2a$ period. Interestingly, however, a 2D linearly dispersive magnetic mode and a quasi-static spin component exist in the low temperature limit. Possibly relevant scenarios including topological phase transition associated with $Z_{2}$ vortex due to vector spin chirality, spin quadrupolar nematic correlation, and $C_{3}$ bond-ordering are reviewed.