unconventional pairing and electronic dimerization ... · doi: 10.1103/physrevb.90.045135 pacs...

PHYSICAL REVIEW B 90, 045135 (2014)

Unconventional pairing and electronic dimerization instabilities in the dopedKitaev-Heisenberg model

Daniel D. Scherer,1,* Michael M. Scherer,2 Giniyat Khaliullin,3 Carsten Honerkamp,4 and Bernd Rosenow1,5

1Institut fur Theoretische Physik, Universitat Leipzig, D-04103 Leipzig, Germany2Institut fur Theoretische Physik, Universitat Heidelberg, D-69120 Heidelberg, Germany

3Max-Planck-Institut fur Festkorperforschung, D-70569 Stuttgart, Germany4Institute for Theoretical Solid State Physics, RWTH Aachen University, D-52056 Aachen, Germany

and JARA - FIT Fundamentals of Future Information Technology, Germany5Physics Department, Harvard University, Cambridge, Massachusetts 02138, USA

(Received 1 May 2014; revised manuscript received 30 June 2014; published 28 July 2014)

We study the quantum many-body instabilities of the t-JK-JH Kitaev-Heisenberg Hamiltonian on thehoneycomb lattice as a minimal model for a doped spin-orbit Mott insulator. This spin-1/2 model is believed todescribe the magnetic properties of the layered transition-metal oxide Na2IrO3. We determine the ground stateof the system with finite charge-carrier density from the functional renormalization group (fRG) for correlatedfermionic systems. To this end, we derive fRG flow equations adapted to the lack of full spin-rotational invariancein the fermionic interactions, here represented by the highly frustrated and anisotropic Kitaev exchange term.Additionally employing a set of the Ward identities for the Kitaev-Heisenberg model, the numerical solution ofthe flow equations suggests a rich phase diagram emerging upon doping charge carriers into the ground-statemanifold (Z2 quantum spin liquids and magnetically ordered phases). We corroborate superconducting tripletp-wave instabilities driven by ferromagnetic exchange and various singlet pairing phases. For filling δ > 1/4,the p-wave pairing gives rise to a topological state with protected Majorana edge modes. For antiferromagneticKitaev and ferromagnetic Heisenberg exchanges, we obtain bond-order instabilities at van Hove filling supportedby nesting and density-of-states enhancement, yielding dimerization patterns of the electronic degrees of freedomon the honeycomb lattice. Further, our flow equations are applicable to a wider class of model Hamiltonians.

DOI: 10.1103/PhysRevB.90.045135 PACS number(s): 71.70.Ej, 74.20.Rp, 71.27.+a

I. INTRODUCTION

The postulation of new topological states of matter andthe quest for unraveling their properties has spurred a hugeamount of research activity over the last decade, building onthe ground breaking insight that the appearance of the integerquantum Hall effect (IQHE) [1,2] in two-dimensional electrongases subject to quantizing perpendicular magnetic fields isintimately related to the topology of wave functions [3–5]. Theprediction of the existence of a topological state generated bythe presence of spin-orbit coupling in HgTe/CdTe quantumwells [6,7] effectively opened out in the creation of the fieldof topological insulators and superconductors [8–12]. Whilethe IQHE still serves as a time-honored prototype to the field,many different models have been identified featuring the sameor similar kinds of topological nontriviality in their wave-functions [8–12].

Another subfield of condensed matter physics with strongconnections to topology is the search for quantum spin liquids[13]—nonmagnetic phases with typically exotic excitationsand topological order [14,15], a concept first introduced in thecontext of the fractional quantum Hall effect (FQHE) [16–19].On the theoretical side, the honeycomb Kitaev model [20]provided the first exactly solvable model with a quantum spinliquid ground state in two spatial dimensions. Though a purespin model, its excitations are Majorana fermions obeyingnon-Abelian exchange statistics [20].

*[email protected]

A route to explore Kitaev physics in a transition-metal oxidesolid-state system, however, was suggested only recently.The naturally large spin-orbit and crystal-field energy scalesin layered transition-metal oxides and the rather strongcorrelation effects in 5d orbitals lead to highly anisotropicexchange interactions, entangling spin and orbital degrees offreedom. As a candidate compound, the layered honeycombiridate Na2IrO3 was proposed [21,22]. It turned out to ordermagnetically below TN ≃ 15 K in a so-called zigzag patterndifferent from the Neel state on the bipartite honeycomblattice [23–31]. Such a low ordering temperature was furthertaken as a sign of strongly frustrated exchange—a hallmarkof Kitaev physics. Indeed, an effective (iso-)spin 1/2 modelfor the two states in the jeff = 1/2 part of the spin-orbit splitmanifold of t2g electrons was suggested to capture the magneticproperties of the Mott-insulating ground state. It is simplygiven by Kitaev exchange and isotropic Heisenberg exchangefor (iso-)spin 1/2 degrees of freedom on nearest-neighborsites in the honeycomb lattice. As the character of exchangeinteractions is varied among the possibilities of ferromagneticand antiferromagnetic couplings, the model features a ground-state manifold with several interesting magnetic orderings withcharacteristic imprints on the spin-wave excitation spectrum.The Z2 quantum spin liquid ground state occurs for eitherferromagnetic or antiferromagnetic Kitaev exchange, as longas the perturbation due to the Heisenberg coupling remainssufficiently small.

While the adequacy of the Kitaev-Heisenberg Hamiltonianas a minimal model for the magnetic properties of thehoneycomb iridate Na2IrO3 is still subject to debate [32], theidea of studying the unconventional pairing states of the doped

1098-0121/2014/90(4)/045135(21) 045135-1 ©2014 American Physical Society

http://dx.doi.org/10.1103/PhysRevB.90.045135

DANIEL D. SCHERER et al. PHYSICAL REVIEW B 90, 045135 (2014)

system has attracted additional attention from theory [33–36].The resulting Hamiltonian can be understood as a paradigmaticmodel of a spin-orbit coupled, frustrated and doped Mottinsulator. Besides singlet pairing phases, mean-field studiesrevealed p-wave triplet pairing phases [33–35], similar to theB phase in 3He. Finally, when doping the system beyondquarter filling, the p-wave (mean-field) states are guaranteed[10,11,37–39] to undergo a transition to a topological p-wavetriplet phase [33–35]. This argument applies at least withinweak-pairing theory. Almost in reminiscence of the Majoranaexcitations of the Z2 quantum spin liquid of the pure Kitaevmodel, the topological p-wave phase would posses Majoranastates at vortex cores and Majorana edge modes propagatingalong the boundaries of the system. Amazingly, this lineof investigation unveils the possible unification of the twodifferent branches of topological insulators/superconductorsand topologically ordered phases in the phase diagram of aparadigmatic single model Hamiltonian.

Regarding the possibility to experimentally realize dopedstates of the honeycomb iridate Na2IrO3, we would liketo mention that electron doping of this material has beenachieved recently by covering the sample with a potassiumlayer [40]. Further, doping of Sr2IrO4 iridium compoundsbecame possible by the same technique and pseudogap physicsas was observed in cuprates [41]. Studying minimal models ofdoped spin-orbit Mott insulators therefore seems a worthwhileenterprise with possible connections to future experiments.

In this work, we investigate the phase diagram of the dopedKitaev-Heisenberg model beyond mean-field theory. This aimis accomplished within the flexible flow equation approachprovided by the functional renormalization group [42] (fRG).

II. MODEL, RESULTS AND OUTLINE

In the following, we introduce the Hamiltonian of thedoped Kitaev-Heisenberg model, see Sec. II A. Our overallgoal is to draw the ground-state phase diagram of the system,paying attention to different, competing ordering tendencies.We summarize and describe the gist of our results in Sec. II B,demonstrating the richness of the phase diagram borne bydoping of the Kitaev-Heisenberg system. We then provide anoutline of this work in Sec. II C. Details are then provided inthe subsequent sections.

A. The doped Kitaev-Heisenberg model

We study the Hamiltonian (1) on the honeycomb latticeas a minimal model for a doped spin-orbit Mott insulator.The Hamiltonian includes a kinetic part, which describes thehopping of the electrons, a Kitaev coupling JK, describinga bond-dependent Ising-like spin exchange, and a Heisen-berg exchange, including a nearest-neighbor density-densityinteraction of magnitude JH that occurs upon doping. TheHamiltonian for the doped spin-orbit Mott insulator thus reads

H = Hkin + HK + HH, (1)

FIG. 1. (Color online) Three plaquettes on the honeycomb lat-tice. White disks mark sites of the A sublattice, while black disks thoseresiding in the B sublattice. Shown are also the three nearest-neighborvectors δi , i = 1,2,3 pointing from a site in the A sublattice to itsnearest neighbors in the B sublattice. The different colors of the bondslinking the sites of the honeycomb encode the bond-specific nearest-neighbor interaction of the Ising-like Kitaev exchange entering theKitaev-Heisenberg Hamiltonian. Red bonds are called x bonds andcorrespondingly, along an x bond the x components of neighboringspin operators are exchange-coupled. The conventions for y and z

bonds are analogous.

where1

Hkin = −t0 P∑

σ,r∈A,δi

(c†A,σ,r cB,σ,r+δi

+ H.c.)P

−µ∑

σ,o∈A,B,r

c†o,σ,r co,σ,r , (2)

HK = JK

∑

r∈A,δi

Sγr S

γ

r+δi, (3)

HH = JH

∑

r∈A,δi

(Sr · Sr+δi

− 14nrnr+δi

). (4)

The kinetic part of the Hamiltonian Hkin describes spin-independent nearest-neighbor hopping of electrons with hop-ping amplitude t0, while the chemical potential µ is adjustedto yield a charge concentration corresponding to the dopinglevel. The Gutzwiller projection P enforces the constraint ofno doubly occupied sites, incorporating the strong correlationeffects of the Mott insulating state. Due to the two-atom unitcell of the honeycomb lattice, the kinetic term leads to atwo-band description for mobile charge degrees of freedom.The sites within the two-dimensional bipartite honeycomblattice are labeled by r . For fixed r in sublattice A, thereare three nearest-neighbor sites within the B sublattice whoseposition is given by r + δi with i ∈ {1,2,3}, cf. Fig. 1. Thenearest-neighbor vectors δi are given by δ1 =

√3a2 ex + a

2 ey ,

δ2 = −√

3a2 ex + a

2 ey , and δ3 = −aey , with a being the distancebetween two neighboring lattice sites and the vectors pointfrom the A to the B sublattice. The operators co,σ,r and c

†o,σ,r

describe annihilation and creation of an electron at position r ina sublattice o ∈ {A,B} with σ = ↑,↓ the isospin polarization,respectively. For simplicity, we will refer to σ as “spin” in theremainder of the paper. We note that generally summation runs

1Here, we defined the exchange couplings JH and JK differentlycompared to Ref. [33].

045135-2

UNCONVENTIONAL PAIRING AND ELECTRONIC . . . PHYSICAL REVIEW B 90, 045135 (2014)

over one sublattice only (thus counting every nearest-neighborbond only once), while the sum in the local chemical-potentialterm runs over either sublattice A or B, depending on whethero = A or B.

Starting from a model with local interactions, within astrong-coupling expansion [43], virtual charge excitationsabove the Mott-Hubbard gap create effective spin-spin in-teractions. Due to large spin-orbit effects and low-symmetrycrystal fields as in the iridates, the exchange interactionsare highly anisotropic. The so-called Kitaev interaction HKdescribes a bond-dependent Ising-like spin exchange, see therightmost plaquette in Fig. 1. Its strength is described by theKitaev coupling JK. For i ∈ {1,2,3} running over the adjacentnearest-neighbor sites residing in the B sublattice, γ ≡ γ (i)takes on the values γ ∈ {x,y,z}. Besides the Kitaev term, themodel contains an additional Heisenberg exchange HH withHeisenberg coupling constant JH, where a nearest-neighbordensity-density interaction due to doping is included. Wenote that the presence of the density-density term allowsto reformulate the Heisenberg exchange solely in terms ofbond-singlet operators. The Hamiltonian (1) gives rise to at-JK-JH model description of a doped spin-orbit Mott insulator.

B. Main results

In this work, we consider the case of ferromagneticKitaev (JK < 0) and antiferromagnetic (JH > 0) Heisenbergexchanges, as well as antiferromagnetic Kitaev (JK > 0)and ferromagnetic (JH < 0) Heisenberg exchanges. Whilethe former realizes a magnetic phase with stripy antifer-romagnetic order (alternating ferromagnetic stripes that arecoupled antiferromagnetically) at δ = 0 in the pure spinmodel, the latter case brings about magnetic order in azigzag pattern (ferromagnetic zigzag chains that are coupledantiferromagnetically) [22–31], see Fig. 2. The Z2 quantumspin liquid exists for both ferromagnetic and antiferromagneticKitaev exchanges for sufficiently small Heisenberg exchangestrength. When both exchange couplings come with the samesign, the magnetic ordering pattern is either of ferromagneticor Neel type.

Motivated by an estimate of the energy scale of the Kitaevexchange relative to the spin-independent nearest-neighborhopping [27], and in order to reduce the number of parametersin the model, we restrict our attention to |JK|/t0 = 1. Asnoted previously [33], at fixed doping, the ratio |JK|/t0 largelycontrols the overall scale for critical temperatures, while theratio |JK|/|JH| determines the ground state of the doped model.

The numerical solutions to the fRG equations adapted tothe Kitaev-Heisenberg model allow us to identify the leadingFermi surface instabilities of an auxiliary fermion system, tobe presented in Sec. III. We introduce the fRG equations insome detail in Sec. IV. Additionally, we can read off estimatesof ordering scales or critical temperatures from the fRG flowsthrough the critical scale $c, at which an instability manifestsas a divergence in the scale-dependent effective interaction. Inthe following, we will present the phase diagrams of the dopedKitaev-Heisenberg model for JK < 0, JH > 0, and JK > 0,JH < 0 as our main results. A detailed discussion of our resultscan be found in Sec. V.

FIG. 2. (Color online) Cuts in the ground-state manifold of theundoped Kitaev-Heisenberg model in the two-dimensional parameterspace spanned by Kitaev (JK) and Heisenberg (JH) exchangecouplings. In our work on the doped case, we restrict our attention tofixed JK. To facilitate a comparison to the undoped system, we chosethe bare hopping as the overall energy scale and plot the ground-statesfor fixed Kitaev exchange as the magnitude of Heisenberg exchangeis varied. For the full phase diagram in the (JK,JH) plane, we referto Ref. [22]. The first row shows the magnetic ground-state patternson the honeycomb lattice, where red and black arrows denote spinup and spin down with respect to a given quantization axis. Thesecond row gives the corresponding parameter ranges. The numbersbelow indicate the critical coupling strengths, where transitions fromone phase to another occur. The Z2 quantum spin liquid (whichoccurs for both JK > 0 and JK < 0) is destabilized by some amountof Heisenberg exchange and turns into either a magnetically orderedzigzag pattern or into an ordered stripy phase. For large ferromagneticHeisenberg exchange, the ground state is rendered ferromagnetic,while for large antiferromagnetic Heisenberg exchange, one finds aNeel antiferromagnet.

1. Ferromagnetic Kitaev, JK < 0, and antiferromagneticHeisenberg, JH > 0, exchanges

We start by considering the Kitaev-Heisenberg modelwith ferromagnetic Kitaev, JK < 0, and antiferromagneticHeisenberg coupling, JH > 0. Figure 3 shows the phasediagram obtained from our fRG instability analysis in theparameter space spanned by doping δ and Heisenberg ex-change JH in units of the bare hopping amplitude t0. TheKitaev coupling is fixed to JK/t0 = −1 in units of the barehopping. The general structure of the phase diagram becomesapparent already at small doping. A p-wave triplet supercon-ductor is supported by ferromagnetic Kitaev exchange. Thesuperconducting instability switches from triplet to singletas the antiferromagnetic Heisenberg coupling increases andfinally gives way to a Neel antiferromagnet, expected for largeJH. Even at low doping, we find no hint of the stripy phase,one of the ground-states encountered in the undoped system,cf. Fig. 2. Although our method becomes unreliable in thelimit δ → 0 due to truncation and additional approximations(see beginning of Secs. V and VI), we interpret this observationas a destabilization of the stripy phase due to finite doping. Fordoping level δ > 1/4, we confirm topological p-wave phasesin the doped Kitaev-Heisenberg model. Further, the mean-fieldprediction of a topological triplet p-wave superconductor withtopologically protected Majorana modes is left untouched [33]by our findings. Our results further suggest that the generationof triplet pairing instabilities in this parameter regime hingessolely on a finite ferromagnetic Kitaev exchange.

045135-3


FIG. 3. (Color online) The phase diagram as obtained from thenumerical solution of a (N = 24)-patching scheme with full fRGwith JK/t0 = −1. Black dots mark the parameters for which fRG-flows were evaluated. The horizontal axis gives the strength of theantiferromagnetic Heisenberg coupling JH > 0 in units of the barehopping amplitude t0, while the doping level δ is given on the verticalaxis. The color code describes the magnitude of the critical scale$c across the phase diagram. The dashed black line marks thetransition to a topological odd-parity pairing state across the vanHove singularity. The inclusion of particle-hole fluctuations onlyaffects the singlet instability. The d-wave instability (d-SC) is turnedinto a magnetic instability with a momentum structure correspondingto a Neel antiferromagnet (AF). For superconducting instabilities,labels in the phase diagram refer to intraband pairing symmetries.See Sec. V A for a detailed description.

Since in the quest for realizing accessible p-wave supercon-ductors, a high transition temperature into the superconductingstate is desirable, let us note that we identify a window ofcritical scales corresponding to p-wave instabilities in therange 10−2t0 to 10−4t0 in units of the bare hopping.

2. Antiferromagnetic Kitaev, JK > 0, and ferromagneticHeisenberg, JH < 0, exchanges

In the case of antiferromagnetic Kitaev, JK > 0, andferromagnetic Heisenberg exchange, JH < 0, we again findtriplet p-wave solutions, which are supported by both Kitaevand Heisenberg exchange. While ferromagnetic Heisenbergexchange polarizes the electronic states and eases the for-mation of Cooper pairs in the triplet channel, the antiferro-magnetic Kitaev interaction turns out to still play a vital rolein forming the instability. Indeed, for JK = 0 and below adoping-dependent critical Heisenberg coupling JH, we observeno ordering tendencies down to the lowest scales accessiblewithin our approach in neither singlet nor triplet channels withδ = 0 (an exception is the special filling δ = 1/4, see below).The phase diagram we obtained in the plane of doping δ andHeisenberg exchange |JH| is shown in Fig. 4 for the case of aconstant Kitaev interaction JK/t0 = 1.

FIG. 4. (Color online) The phase diagram as obtained fromthe numerical solution of a (N = 24)-patching scheme with fullfRG with JK/t0 = 1. The phase boundaries were checked with a(N = 96)-patching scheme, and only the singlet SC to triplet SCphase boundary was mildly revised. Black dots mark the parametersfor which fRG flows were evaluated. The horizontal axis gives thestrength of the ferromagnetic Heisenberg coupling JH < 0 in units ofthe bare hopping amplitude t0, while the doping level δ is given on thevertical axis. The color code describes the magnitude of the criticalscale $c across the phase diagram. The dashed magenta line marks thevan Hove singularity. The magenta shading represents the formationof charge (cBO) and spin bond-order (sBO) instabilities at van Hovefilling, as obtained from a (N = 96)-patching scheme. For the criticalscales at van Hove filling, see Fig. 16. For details on bond order, seeSec. V C. The p-wave triplet phase appears in a parameter rangewhere ferromagnetic exchange interactions dominate. Labels forsuperconducting instabilities refer to intraband pairing symmetries.For large Heisenberg exchange, the instability is of CDW type due topartice-hole fluctuations. See Sec. V B for a detailed description.

At small doping and Heisenberg exchange, we find theground state to realize a Neel antiferromagnet. Increasingthe strength of the Heisenberg interaction while keepingthe doping level low, a narrow superconducting window issandwiched between the Neel state and a charge density wavestabilized by the density-density contribution to Eq. (1). Thissuperconducting region increases in size as the doping grows.The ferromagnetic Heisenberg exchange naturally favors thep-wave triplet over singlet pairing, and we observe an extendedregion of triplet p-wave pairing throughout the phase diagram.A topological p-wave state hosting Majorana edge-excitationsis formed for doping δ > 1/4. The range of critical scales thatserves as an estimate of transition temperatures, is here givenas 10−1t0 to 10−8t0.

For the special case of van Hove filling δ = 1/4, however,we encounter a family of instabilities not present for ferro-magnetic Kitaev and antiferromagnetic Heisenberg exchange,see dashed magenta line in Fig. 4. At δ = 1/4, the Fermisurface becomes straight, leading to nesting and enhancedDOS effects. As expected, these conditions strongly enhance

045135-4


the tendency for particle-hole instabilities. For antiferromag-netic Kitaev and ferromagnetic Heisenberg exchange, weobserve bond-order instabilities beyond a rather small criticalHeisenberg coupling |JH|/t0 ≃ 0.2. In fact, even for JK = 0,we observe the formation of bond-order instabilities at vanHove filling. The bond-order instability is leading until wehit the triplet p-wave/charge-density wave phase-boundary.Once the pairing neighborhood in the phase diagram hasdisappeared, bond-order signatures in the vertex functionare rendered subleading. No leading bond-order instabilityis found for ferromagnetic Kitaev and antiferromagneticHeisenberg exchange. Our results add to the current surgeof unconventional bond-order instabilities with concomitantelectronic dimerization in triangular lattice systems withnontrivial orbital or sublattice structure and longer-rangedinteractions. We find, however, only nearest-neighbor dimer-ization, which rules out the possibility of a dynamicallygenerated topological Mott insulator, that has previouslybeen found for extended single- and multilayer honeycombHubbard models. Exotic properties of previously identifiedbond-order phases include charge fractionalization due tovortices in the Kekule order [44,45], valence-bond crystalstates [46], spontaneously generated current patterns [45],and interaction-driven emergent topological states [45,47–51].One other remarkable feature of the dimerized phase with spinbond order of the Kitaev-Heisenberg model is the dynamicalregeneration of spin-orbit coupling. A similar observationwas made in the Kagome Hubbard model with fRG methods[52]. Interestingly, in the present case, it does not lead toa gapped state, but instead we obtain a metallic state in adownfolded Brillouin zone. We note that bond order is notexclusive to systems with an underlying triangular lattice orcomplex orbital structure, but is also observed in square latticesystems [53–56].

C. Outline of this paper

In Sec. III, we derive an auxiliary fermion Hamiltonian onthe honeycomb lattice that will form the input to our fRGcalculations. To account for doping effects in the Kitaev-Heisenberg system, we start with a t-JK-JH model. The kineticenergy will be minimally described by spin-independenthopping, assuming that high-energy spin-orbit effects areaccounted for by the Kitaev exchange term. The exclusionof doubly occupied sites due to the strong interactions inthe Mott insulating state, i.e., a Gutzwiller projection, isdealt with by the slave-boson method. In Sec. III A, wediscuss the slave-boson treatment of the Mott insulating stateand the mean-field approximations in the bosonic sector tomap the problem onto a metal of auxiliary fermions withrenormalized Fermi surface coupled by exchange interactions.The resulting Hamiltonian is then split into singlet andtriplet bond-operators. We then briefly recapitulate slave-boson mean-field results obtained previously by variousgroups in Sec. III B. In Sec. IV, we provide the necessarybackground of the functional renormalization group method.It is based on the idea of the Wilsonian renormalizationprocedure of treating quantum corrections to the classicalsector of a given theory by successively integrating outhigh-energy momentum shells, renormalizing the vertices of

the remaining low-energy degrees of freedom. It relies on anexact hierarchy of coupled renormalization group equationsfor the one-particle irreducible vertex functions. A solution ofour flow equations corresponds to an unbiased resummation ofone-loop diagrams in both particle-particle and particle-holechannels. Our results are presented in detail in Sec. V. Thephase diagram for ferromagnetic Kitaev and antiferromagneticHeisenberg exchanges is discussed in Sec. V A. We firstdemonstrate the capability of the fRG method to reproducemean-field results for the pairing channel in Sec. V A 1 bystudying the effect of exclusive particle-particle fluctuations.This is done both at the level of the phase diagram and theform factors of the superconducting order parameters. We thenmove on to include particle-hole fluctuations and discuss theresulting modifications of the phase diagram in Sec. V A 2as compared to the pure particle-particle resummation in thepairing channel. In Sec. V B, we flip the signs of exchangeinteractions and discuss the phase diagram for the dopedKitaev-Heisenberg model with antiferromagnetic Kitaev andferromagnetic Heisenberg exchanges. Section V C is devotedto the rather special filling condition δ = 1/4 with coincidingvan Hove singularity and perfect Fermi surface nesting.Finally, in Sec. VI, we briefly discuss our findings and discussthe validity of applying the fRG method to the t-JK-JH modelwith strong interactions.

III. SLAVE-BOSON FORMULATION FOR A DOPEDAND FRUSTRATED MOTT INSULATOR

In the following, we briefly describe the U(1) slave-bosonconstruction [33] to deal with the Gutzwiller projection ontothe Hilbert space of no doubly occupied sites. This yields aneffective theory in terms of auxiliary fermionic degrees offreedom, where the bosonic charge excitations (holons) havecondensed by assumption. The background holon-condensatewill then give rise to an effective renormalized dispersion forthe auxiliary fermions. This dispersive fermionic model willform the starting point for our fRG analysis of the dopedKitaev-Heisenberg model.

A. From Kitaev-Heisenberg t- JK- JH modelto a U(1) slave-boson model

The local electron Hilbert space corresponding to co,σ,r andc†o,σ,r can be represented by fermionic fo,σ,r , bosonic holon bo,r

and doublon do,r degrees of freedom. This description, how-ever, introduces unphysical states. To project the artificiallyenlarged Hilbert space down onto the physical states, the localconstraint

∑

σ

f†o,σ,rfo,σ,r + b

†o,r bo,r + d

†o,r do,r = 1 (5)

needs to be enforced on the operator level. The Gutzwillerprojection is now taken into account by deleting doublonoperators and states from the theory.

The electron creation and annihilation operators are now tobe replaced by

co,σ,r → b†o,r fo,σ,r , c

†o,σ,r → f

†o,σ,r bo,r . (6)

045135-5


Importantly, the spin operator can be written in terms of auxil-iary fermions only, Sr = f

†r ,σ σσσ ′fr,σ ′ , with σ = (σx,σy,σz)T

the vector of Pauli matrices. Assuming the holons to Bosecondense into a collective state, ⟨b†b⟩ = δ, the local constraintfor the fermions becomes

∑σ f

†o,σ,rfo,σ,r = 1 − δ.

The kinetic part of the Hamiltonian can now be cast in theform

Hkin = −t∑

σ,r∈A,δi

(f †A,σ,rfB,σ,r+δi

+ H.c.)

−µf

∑

σ,o∈A,B,r

f†o,σ,rfo,σ,r (7)

with the renormalized hopping amplitude t = t0 δ and µf =δ µ the chemical potential which we adjust such that∑

σ ⟨f †o,σ,rfo,σ,r⟩ = 1 − δ is fulfilled. The constraint responsi-

ble for eliminating the unphysical states is thus only includedon average in this approach.

In going from the tight-binding to the Bloch representationby introducing the operators

fo,σ,k = 1√N

∑

r

eik·r fo,σ,r , (8)

where N is the number of unit cells of the honeycomb lattice,we obtain the Bloch Hamiltonian

Hkin = −∑

σ,k

(f †A,σ,k

,f†B,σ,k

)(

µf td∗k

tdk µf

)(fA,σ,k

fB,σ,k

)(9)

with dk =∑

δieik·δi . The dispersion is analogous to electrons

moving in a graphene monolayer, with the concomitant K andK ′ points in the Brillouin zone.

The interaction terms quartic in the fermion operators canbe expressed in terms of singlet and triplet contributions, whichyields a very convenient starting point for the analysis ofsuperconducting instabilities.

The spin-singlet operator defined on a bond connecting siter ∈ A to r + δi ∈ B is defined as

sr,δi= fA,σ,r [%0]σσ ′ fB,σ ′,r+δi

= 1√2

(fA,↑,rfB,↓,r+δi− fA,↓,rfB,↑,r+δi

) (10)

and correspondingly the x, y, z triplet operators read

tr,δi ;x = fA,σ,r [%x]σσ ′ fB,σ ′,r+δi,

= 1√2

(fA,↓,rfB,↓,r+δi− fA,↑,rfB,↑,r+δi

), (11)

tr,δi ;y = fA,σ,r [%y]σσ ′ fB,σ ′,r+δi

= i√2

(fA,↑,rfB,↑,r+δi+ fA,↓,rfB,↓,R+δi

), (12)

tr,δi ;z = fA,σ,r [%z]σσ ′ fB,σ ′,r+δi

= 1√2

(fB,↑,rfA,↓,r+δi+ fB,↓,rfA,↑,r+δi

). (13)

Here, we used the % matrices %0 = 1√2σ0iσy , %x = 1√

2σx iσy ,

%y = 1√2σy iσy and %z = 1√

2σziσy . See Appendix A 1 for

explicit expressions for these matrices and their relation tosuperconducting order parameters. The interaction part of theslave-boson Hamiltonian

Hslave = Hkin + H(s)int + H

(t)int (14)

can now be recast as a sum of a singlet interaction

H(s)int = −

(JH + JK

4

) ∑

r∈A,δi

s†r ,δi

sr,δi(15)

and a triplet interaction

H(t)int = −JK

4

∑

l∈{x,y,z}

∑

r∈A,δi

ζ lr,δi

t†r ,δi ;l

tr,δi ;l . (16)

Here, the index l runs over the triplet components x, y,and z. The prefactor ζ l

r,δidescribes a bond-dependent sign

modulation of the interaction term for each triplet component:we have ζ l

r,δi= +1 if the bond-type (x, y or z) from site r

to r + δi coincides with the triplet component l. Otherwise,ζ lr,δi

= −1. The highly frustrated Kitaev term contains asinglet contribution, which renormalizes the singlet interactioncoming from Heisenberg exchange. The contribution to thetriplet channel from the Kitaev term is irreducible in the sensethat it does not contain any “hidden” singlet contributions.Thus the singlet-triplet decomposition is unique.

We note that the interaction does not contain terms thatdescribe the decay of a singlet state into a triplet state orvice versa. It is worth emphasizing that we model the kineticterm for the auxiliary fermions with a simple nearest-neighborhopping, which furthermore preserves spin. Thus the onlySU(2)spin violating contribution to the Hamiltonian comesfrom Kitaev exchange.

In a solid-state system with strong spin-orbit coupling, theSU(2)spin symmetry is locked to the point group of the lattice[34,57], i.e., only the simultaneous application of point-grouptransformations and the corresponding representation of thepoint-group operations on the spin degrees of freedom leavethe Hamiltonian invariant. In, e.g., the iridates, the Kitaevterm arises precisely due to the presence of strong spin-orbit coupling. As noted previously [34], it is thus naturalthat the symmetry of the Kitaev term involves simultaneoustransformation of both spin and lattice (or wave vector). Therelevant symmetry transformations in the case of the Kitaevterm (with the same coupling JK on nearest-neighbor bonds)acting on the lattice can be understood as a 2π/3 rotationaround the center of a honeycomb hexagon. To maintaininvariance of the Kitaev Hamiltonian, spins have to be rotatedby the same angle around the axis n = 1√

3(1,1,1)T , where the

coordinate system corresponds to an embedding of a honey-comb layer into a 3D cubic lattice [34]. See Appendix A 4for further details.

B. Comparison with mean-field theory

The phase diagram of the doped Kitaev-Heisenberg modelhas previously been studied within different slave-bosonformulations. If quantum fluctuations were treated exactly

045135-6


in these different formulations, they should yield equivalentresults. In practice, however, the emergent gauge fields inslave-boson approaches for strongly correlated lattice fermionsare treated in a mean-field approximation. The local Hilbert-space constraint is realized only on average. While this is alsotrue in our slave-boson formulation of the t-JK-JH model,where the holon condensation is built into the theory inmean-field fashion, the fermionic fRG takes into accountthe fermionic fluctuations in all channels in an otherwiseunbiased way. To allow for a systematic and self-containedcomparison of our fRG results to mean-field studies reportedin the literature, we briefly summarize recent findings.

1. U(1) slave-boson mean-field theory

A previous self-consistent mean-field study [33] of thedoped Kitaev-Heisenberg model taking into account onlysuperconducting order parameters demonstrated a rich phasediagram upon doping charge carriers into the Z2 quantumspin liquid (QSL) and the Mott insulating stripy phase. Forferromagnetic Kitaev coupling, JK < 0, the QSL state at zerodoping is stable with respect to Heisenberg-type perturbationsfor |JH| < |JK|/8, cf. Ref. [27]. Keeping the value of the Kitaevexchange coupling fixed for the undoped system, the stripyphase is realized for antiferromagnetic Heisenberg exchangecoupling, JH > 0, with JH ≃ 0.25, . . . ,1.5. At finite doping,δ ! 0.1, for small antiferromagnetic Heisenberg coupling anddominating ferromagnetic Kitaev coupling, |JK|/2 > JH, atime-reversal invariant p-wave state (p-SC) was found. Uponincreasing doping in this regime, a topological phase transitionto a topological pairing state with p-wave symmetry in thetriplet channel was obtained [33] in the vicinity of van Hovefilling. This state is stable to interband pairing correlationsand was recently shown to be robust also against weaknonmagnetic disorder [36]. Its topological properties can becharacterized by a nontrivial topological Z2-invariant [33,36]from which we can infer that it falls into the DIII symmetryclass of the Altland-Zirnbauer classification [8,12].

For antiferromagnetic Heisenberg coupling JH ! |JK|/2 atsmall doping and ferromagnetic Kitaev coupling, the leadingpairing correlations occur in the singlet channel with intrabandd-wave symmetry. In the large JH regime, an extended singlets-wave state was reported.

2. SU(2) slave-boson mean-field theory

Within an SU(2) formulation [34] and a specific mean-field Ansatz that reproduces the Z2 quantum spin liquid in theδ = 0 Kitaev limit JH = 0, a time-reversal symmetry breakingp-wave state in the triplet channel (p-SC1) was found upondoping the stripy phase. Interestingly, it supports chiral edgemodes and localized Majorana states in the low-doping regime.The physics of this state was argued to be dominated by itsvicinity to the QSL at JH = 0, while for increased doping afirst-order transition to a BCS-like state (p-SC2) occurs.

A further extensive mean-field study [35] for the dopedKitaev-Heisenberg model utilizing the SU(2) formulation withan Ansatz that includes pairing and magnetic order parameterssupports the previous findings. In both mean-field studiesfor ferromagnetic JK and antiferromagnetic JH couplings,however, time-reversal breaking and time-reversal invariant

p-wave states are reported as (almost) energetically degeneratefor the p-SC2 state. In the time-reversal invariant case, thep-SC2 state obtained from the SU(2) formulation coincideswith the p-SC state from the U(1) slave-boson theory. Forlarge |JH|, again singlet instabilities are obtained, with pairingsymmetry changing from d to s wave upon increasing thedoping level.

For antiferromagnetic JK and ferromagnetic JH, a p-SC1phase that extends to large doping was reported. Roughly,for dominating antiferromagnetic Kitaev coupling, a d-wavesinglet solution was obtained, with a transition to a p-SC2phase upon increasing JH and/or doping. For JH ≫ |JK|, allpairing correlations disappear and a ferromagnetic orderingemerges. Mean-field Ansatze other than superconducting andferromagnetic were not considered.

IV. FRG METHOD

We employ a functional renormalization group (fRG)approach for the one-particle-irreducible (1PI) vertices witha momentum cutoff. For a recent review on the fRG method,see Refs. [42,58]. The actual fRG calculation is performedin the band basis in which the quadratic part of the fermionHamiltonian is diagonal. The free Hamiltonian can be diago-nalized by a unitary transformation of the form

fb,σ,k =∑

o

ubo,k fo,σ,k , f†b,σ,k

=∑

o

u∗bo,k

f†o,σ,k

, (17)

where o ∈ {A,B} labels the two sublattices and the index bdenotes the corresponding bands. This transformation alsoaffects the bare interaction vertex and leads to so-calledorbital make up. Since we do not consider mixing of spinstates by the kinetic term of the Hamiltonian, this unitarytransformation does not involve spin projection. In the bandrepresentation, the propagator is also diagonal, with diagonalentries encoding the dispersion ϵ(k,b) of the various bandslabeled by b. In the standard 1PI fRG scheme we employhere, an infrared regulator with energy scale $ is introducedinto the bare propagator function in the band representationG0(ξ,ξ ′) ∼ δξ,ξ ′ , where the label ξ = (σ,b,ω,k) collects spinprojection σ , band index b, frequency ω, and Bloch momentumk. We thus replace

G0(b,ω,k) → G$0 (b,ω,k) = C$[ϵ(k,b)]

iω − ϵ(k,b). (18)

As the spin quantum number σ carried by the auxiliaryfermion degrees of freedom is conserved by the kinetic partof the Hamiltonian (14), the free propagator is diagonal alsoin spin indices. The cutoff function is chosen to enforce anenergy cutoff, which regularizes the free Green’s function bysuppressing the modes with band energy below the scale $,

C$[ϵ(k,b)] ≈ +(|ϵ(k,b)| − $) . (19)

For better numerical feasibility, the step function is slightlysoftened in the actual implementation. With this modifiedscale-dependent propagator, we can define the scale-dependenteffective action %$ as the Legendre transform of the generatingfunctional G$ for correlation functions, cf. Refs. [42,59].The RG flow of %$ is generated upon variation of $. By

045135-7


integrating the flow down from an initial scale $0, onesmoothly interpolates between the bare action of the systemand the effective action at low energy.

A. Flow equations for SU(2)spin noninvariant systems

While the U(1) slave-boson model (14) is equipped withglobal U(1) symmetry in the fermion sector even afterthe bosonic holons have condensed, there is no SU(2)spinsymmetry present as for electronic systems without spin-orbitcoupling. Since the interactions do not conserve the spinquantum number carried by the fermions, the four-point vertexwill also depend on the specific spin configuration of incomingand outgoing states. Without any further symmetry constraints,this leads to a total of 16 = 24 independent coupling functions,one for each possible spin configuration. For the Hamiltonianin Eq. (14), however, it suffices to consider only four vertexfunctions. These are simply given by singlet-singlet and triplet-triplet interactions. The renormalization group flow preservesthis property, i.e., if it is present in the initial condition,singlet-triplet mixing terms will never be generated duringthe RG evolution.2

This allows us to factor out the spin-indices from thevertex functions. To this end, we introduce the short-hand notation ξ = (σ,ξ ) for the set of quantum numbers,where ξ = (o,ω,k) is in orbital/sublattice representation andξ = (b,ω,k) is in band representation. After factorization,the flow equations can be formulated for only four vertexfunctions depending on the quantum numbers ξ , exclusively.With these premises, the scale-dependent coupling functionV $ = V $(ξ1,ξ2,ξ3,ξ4) can be expanded in terms of the singletvertex function, V (s) = V (s)(ξ1,ξ2,ξ3,ξ4), and the triplet vertexfunctions, V

(t)l = V

(t)l (ξ1,ξ2,ξ3,ξ4), as

V $ = −V (s)[%†0]σ1σ2 [%0]σ3σ4 +

∑

l∈{x,y,z}V

(t)l [%†

l ]σ1σ2 [%l]σ3σ4 .

(20)

The projection onto scale-dependent singlet V (s) and tripletV

(t)l vertex functions can be facilitated with the orthogonality

properties of the % matrices, see Appendix A 1, effectivelytracing out spin quantum numbers σi , i = 1, . . . ,4.

The singlet vertex function V (s) is fully symmetric withrespect to exchanging ξ1 ↔ ξ2 and ξ3 ↔ ξ4. The antisym-metric spin matrix %0 ensures overall antisymmetry underthese exchange operations, as required by a fermionic four-point function. Correspondingly, the triplet vertex functionsV

(t)l are antisymmetric under these exchange operations with

symmetric spin matrices %l .Employing the flow equations appropriate for global U(1)

symmetry [60], we find for the singlet evolution

d

d$V (s) = φ(s)

pp + φ(s)ph,cr + φ

(s)ph,d, (21)

2This property hinges on the SU(2)spin invariance of the kineticHamiltonian. Generally, the fRG flow is expected to leave themanifold spanned by the singlet and triplet vertex functions, if wewere to include spin-dependent hopping processes. Then the fullspace of spin configurations needs to be resolved.

where φ(s)pp , φ

(s)ph,d, and φ

(s)ph,cr denote the particle-particle, direct

particle-hole, and crossed particle-hole RG contributions tothe singlet channel. The triplet evolutions are found along thesame lines as

d

d$V

(t)l = φ

(t)pp;l + φ

(t)ph,cr;l − φ

(t)ph,d;l , l ∈ {x,y,z}, (22)

with the corresponding RG contributions for the three tripletchannels. The scale-dependent bubble contributions appearingon the right-hand sides of Eqs. (21) and (22) are in turnquadratic functionals of V (s) and V

(t)l . The explicit expressions

are summarized in Appendix A 3. From these it followsthat singlet and triplet channels exert a mutual influenceonly through particle-hole fluctuations. For a diagrammaticrepresentation of the flow equations (21) and (22) for singletand triplet vertices, see Fig. 5. Neglecting both direct and

ξ3

ξ4

ξ1

ξ2

(s)/(t,l)

=

(s)/(t,l) (s)/(t,l)

particle-particle contribution

(s)

(s)

+ +

(s)

(t,l)crossed particle-hole

contributions (t,l)

(s)

+ +

(t,l)

(t,l′)

(s)

(s)

± ±

(s)

(t,l)direct particle-hole

contributions (t,l)

(s)

± ±

(t,l)

(t,l′)

FIG. 5. Diagrammatic representation of flow equations (21) and(22). The dot on the l.h.s of the equation denotes the scale derivatived/d$. In the loops, one line always corresponds to a bare regularizedpropagator G$

0 , while the other one corresponds to a so-called singlescale propagator S$ = d/d$G$

0 . The labels (s) and (t,l) distinguishsinglet and triplet vertices. The first contribution to the flow is therespective particle-particle diagram φ(s)

pp , φ(t)pp;l that does not couple

singlet and triplet channels. The remaining two sets of diagramscorrespond to the crossed (φ(s)

ph,cr, φ(t)ph,cr;l) and direct particle-hole

bubbles (φ(s)ph,d, φ

(t)ph,d;l). In the direct particle-hole diagrams, the

positive (negative) signs refer to the contributions to the flow ofthe singlet (triplet) vertex. See also Appendix A 3 for details on thedifferent sign structures for singlet and triplet flow equations.

045135-8


crossed particle-hole contributions to the RG evolutions ofsinglet and triplet coupling functions, Eqs. (21) and (22)decouple so that singlet and triplet vertex functions evolveindependently. This decoupling of singlet and triplet channelsis also found in BCS-like mean-field theory from a linearizedgap equation. We thus expect that the inclusion of onlythe particle-particle bubbles φ(s)

pp and φ(t)pp;l as driving forces

in the flow equations yields a resummation that reproducesmean-field results. Details are discussed in Sec. V A.

We note there is a difference in sign between crossed anddirect particle-hole contributions in the singlet and triplet flowequations, Eqs. (21) and (22), respectively . This sign reflectsthe exchange symmetries of the various vertex functions, suchthat the incremental changes dV (s) and dV

(t)l come with the

correct (anti-)symmetrization, since neither crossed nor directparticle-hole bubbles have the symmetry property on their own.

B. Approximations and numerical implementation

In order to limit the numerical effort, we employ a numberof approximations. First, the hierarchy of flowing vertex func-tions is truncated after the four-point (two-particle interaction)vertex. Second, we employ the static approximation, i.e., weneglect the frequency dependence of vertex functions, bysetting all external frequencies to zero, as we are interestedin ground-state properties. Third, self-energy corrections areneglected. This approximate fRG scheme then amounts toan infinite-order summation of one-loop particle-particle andparticle-hole terms of second order in the effective interactions.It allows for an unbiased investigation of the competitionbetween various correlations, by analyzing the componentsof V (s) and V

(t)l that create instabilities by growing large

at a critical scale $c [42]. From the evolving pronouncedmomentum structure, one can then infer the leading orderingtendencies. With the approximations mentioned above, thisprocedure is well controlled for small interactions. At interme-diate interaction strengths, we still expect to obtain reasonableresults and it was recently shown that the fRG-flow producessensible results even in proximity to the singularity [61]. Inany case, the fRG takes into account effects beyond mean-field and random phase approximations. This way, the fRGrepresents an alternative to the inclusion of gauge fluctuationsin the mean-field theories. The wave vector dependence ofthe interaction vertices is simplified by a discretization—theN -patch scheme—that resolves the angular dependence alongthe Fermi surface for a given chemical potential. The Brillouinzone (BZ) is divided into N patches with constant wave-vectordependence within one patch, so that the coupling functionhas to be calculated for only one representative momentum ineach patch. The representative momenta for the patches arechosen to lie close to the Fermi level. The patching schemeis shown in Fig. 6 for N = 24. Calculations were performedfor different but fixed angular resolution with N = 24 and48 as well as N = 96 to check the reliability of the resultswith respect to higher resolution. The vertex functions furtherdepend on sublattice or band labels. Since overall momentumconservation leaves only three independent wave vectorsin the BZ, a single vertex function is approximated by a24×N3 component object. In total, we thus obtain 4×24×N3

coupled differential equations for the approximated singlet and

K 'K '

KK

1 23

4 5

67 8

9

10

11

12

131415

161718

1920

21

22

23

24

FIG. 6. (Color online) Patching of the Brillouin zone (BZ) withN = 24 for different doping levels. For µ = 0, the representativepatch momenta (colored dots) move toward the % point in the BZcenter. At |µ| = t0, the Fermi surface segments become “straight,”i.e., perfect nesting is realized, which is reflected in a van Hovesingularity in the density of states. For |µ| ! t0, the actual imple-mentation of the patching needs to be slightly modified. The red,green, and blue lines correspond to the Fermi surfaces for dopingδ ≈ 0.08, 0.14, and 0.27, respectively.

triplet vertex functions. Exploiting the implications of residualrotational symmetry as outlined above (see also Appendix A 4)for vertex reconstruction in the triplet channel, this number canbe reduced by a factor of 2.

V. ORDERING TENDENCIES FROMFUNCTIONAL RG FLOWS

We start the fRG flow at the initial scale $0, which wechoose as the largest distance in energy from the location of theFermi surface to the lower and top band edges of valence andconduction bands, respectively. By solving the flow equationsnumerically, we successively integrate out all modes of thesebands in energy shells with support peaked around the RG-scale $. In the case of an instability, some components ofthe scale-dependent effective interaction vertices become largeand eventually diverge at a critical scale $c > 0. Since the flowneeds to be stopped at a scale $∗ " $c, we take as a stoppingcriterion the condition that the absolute value of one of thevertex functions exceeds a value of the order of 100 timesthe bare bandwidth t0. We further assume that $∗ ≃ $c. Theprecise choice for the stopping criterion affects the extractedvalue for the critical scale very mildly, as the couplings growvery fast in the vicinity of the divergence.

In our analysis, we kept the value of the bare hopping t0fixed, and also fixed the value for the Kitaev coupling JK, whilethe doping δ and Heisenberg coupling JH are varied.

To elucidate the role of holon condensation on the systemparameters, cf. Eq. (7), we consider the auxiliary fermionHamiltonian (14), and its corresponding partition functionZ(β) = Tr e−βHslave . While doping levels δ < 1 reduce thebandwidth of the fermion system, we can equivalently viewthis as a renormalization of the bare exchange interactions.Since the partition function Z(β) is invariant upon rescaling

045135-9


temperature as β → βδ and at the same time rescalingthe Hamiltonian Hslave → H ′

slave = Hslave/δ, the ground-stateproperties of Hslave in the limit β → ∞ can be extracted fromH ′

slave.In total, this corresponds to rescaling the hopping ampli-

tude, chemical potential, and couplings as

t → t

δ= t0, µf → µf

δ= µ, JH → JH

δ, JK → JK

δ.

(23)

The rescaling entails large absolute values for the vertexfunctions already in the initial condition at least in thelow-doping regime δ ≃ 0.1. While we can conveniently keepthe kinetic energy scale at t0, we cannot set the bar forthe stopping criterion too low on the vertex functions toallow—at least—for a sizable evolution along the $ direction.From these considerations, it is also immediately obviousthat within our slave-boson approach and the employedapproximations to the exact hierarchy of flow equations, wecannot describe the magnetic instabilities of the Mott insulatoras δ → 0. Including the flowing self-energy .$ and keepingthe frequency dependence in the flows for self-energy andeffective interaction vertex, we could expect to bridge thedescription to the Mott insulating state, cf. Sec. VI andRefs. [26,29,62].

A divergence in the interaction vertex can be consideredas an artefact of our truncation, which as it stands completelyneglects self-energy feedback. Here, we thus restrict ourselvesto an analysis of leading ordering tendencies at finite doping.The pronounced momentum structure of a vertex functionclose to the critical scale $c can be used to extract aneffective Hamiltonian for the low-energy degrees of freedom,which can, in principle, be decoupled by a suitable Hubbard-Stratonovich field that is then dealt with on a mean-field level.This is used to determine the order parameter correspondingto the leading instability. Furthermore, the scale $c canbe interpreted as an estimate for ordering temperatures, ifordering is allowed, or at least as the temperature below whichthe dominant correlations should be clearly observable.

A. Doping QSL and stripy phase: FM Kitaev and AFHeisenberg exchanges

We first consider the case of a ferromagnetic Kitaev, JK <0, and antiferromagnetic Heisenberg coupling, JH > 0. Atdoping level δ = 0, there exists an extended region in the spaceof couplings JH, JK where the stripy phase is realized as themagnetically ordered ground state of the strongly correlatedspin-orbit Mott insulator. In the following, we will fix theferromagnetic Kitaev coupling as JK/t0 = −1. The range ofHeisenberg couplings we focus on is given by JH/t0 ∈ [0,1.5].This parameter range includes the full extent of the stripyphase at δ = 0, and also for small JH, the dominating Kitaevterm is responsible for the realization of the quantum spinliquid phase (QSL). However, here we focus on a dopingregime δ ∈ [0.1,0.5], where the effects of the proximity tothe QSL phase are not visible any more. We thus cannotobserve a signature of the p-SC1 state [34,35], cf. Sec. III B.Slave-boson mean-field studies connecting to the Kitaev limit

as δ → 0 suggest [34,35] that holon condensation sets inrapidly at JH = 0 as δ is increased from 0 to a small butfinite value. Although this seems to render our Hamiltonian asensible starting point from the point of view of mean-fieldtheory, the small renormalized bandwidth ∼t yields hugerescaled couplings, turning the fRG-flow unreliable. In thedoping regime δ < 0.1, a pure particle-particle resummationdetects no sign of a first-order transition between two differenttriplet p-wave phases. While upon the inclusion of partice-holefluctuations the superconductivity seems to disappear, thedivergent vertex functions do not yield a clear picture as to whatkind of instability is actually realized. As a consistency checkon our flow equations, we first completely neglect crossed anddirect particle-hole bubbles in the flow. From a diagrammaticperspective, the numerical solution of the flow equations isexpected to reproduce the results of a mean-field analysis ofthe slave-boson Hamiltonian with only superconducting orderparameters. Second, we obtain the phase diagram for the dopedstripy phase with particle-particle and particle-hole bubblesincluded on equal footing in the flow equations.

1. Resummation in particle-particle channel

As already mentioned in Sec. IV, only particle-hole fluctu-ations couple singlet and triplet vertex functions among eachother. Neglecting particle-hole bubbles amounts to taking intoaccount only the first diagram in Fig. 5 in the flow equation.We find that the leading instability is essentially determinedby the structure of the initial condition for the flow equation.For a dominating singlet vertex, the flow leads to an increasein the singlet channel, where also the dominating pairingsymmetry of the initial condition is enhanced. The amplitude ofthe subleading pairing solutions is not substantially increasedby the fRG-evolution. For a superconducting instability, theassociated spin structure of the leading pairing correlations canbe immediately inferred from whether the singlet vertex V (s) orthe triplet vertices V

(t)l diverge. Due to the residual rotational

symmetry of the Kitaev-Heisenberg model (see Sec. IV andAppendix A 4), the three triplet vertex functions are bound todiverge simultaneously. This also leads to a degeneracy forthe pairing solutions for the d vector describing the structureof the corresponding Cooper pair. In order to obtain suchinformation from the vertex functions, we extract the pairscattering amplitudes in singlet and triplet channels as

V (s)(k,k′) ≡ V (s)(k, − k,k′, − k′), (24)

V(t)l (k,k′) ≡ V

(t)l (k, − k,k′, − k′), (25)

where for brevity we suppressed sublattice or band labels.Since our discretization of the Brillouin zone employs atotal of N representative patch momenta, the pair-scatteringamplitudes can be treated as N×N matrices. Diagonalizationof V (s)(k,k′) or V

(t)l (k,k′) and determination of the eigenvectors

vk,λ corresponding to the eigenvalues λ unveils the leadingand subleading pairing instabilities [63–66]. The two-particlecontributions to the effective action %$c at the critical scale$c determined by the leading instability (i.e., the eigenvalueλ with largest absolute value) in, e.g., the singlet channel

045135-10


become

H(s)SC ∝ −λ

∑

k,k′

(v∗k,λ

f†k%†0f

†−k

)(vk′,λfk′%0f−k′), (26)

where sublattice/band and spin labels were again suppressedfor clarity. The Hamiltonian H

(s)SC can be decoupled by a

Hubbard-Stratonovich transformation with a singlet order-parameter field ψk ∼ vk,λ⟨fk%0f−k⟩. Analogous definitionshold for the triplet case, where the decoupling is performedwith the vector order parameter dk , cf. Appendix A1. Theorder-parameter symmetry, i.e., the momentum-space Cooperpair structure, is obtained by projecting the eigenvectors vk,λ

onto suitably defined form factors. These can be obtainedfrom the irreducible representations of the point group ofthe hexagonal lattice in real space [57,65]. From a neighbor-resolved Fourier transform one can obtain momentum-spacerepresentations of form factors with well-defined parity. SeeAppendix A2 for details. The phase diagram as obtained froman analysis of the leading instabilities is shown in Fig. 7.The overall structure of the phase boundary between intrabandp-SC and singlet pairing phases agrees nicely with the findingsin Ref. [33]. While Ref. [33] reports a singlet s-wave regimethat extends to low doping, the singlet s-wave instabilityappears only for δ > 0.4 in our fRG calculations. This findingagrees with the phase diagram obtained in Ref. [67], wherethe triplet channel was neglected. We interpret this in favor

FIG. 7. (Color online) The phase diagram as obtained from thenumerical solution of a (N = 24)-patching scheme with only particle-particle bubbles with JK/t0 = −1. Black dots mark the parametersfor which fRG-flows were evaluated. The horizontal axis gives thestrength of the antiferromagnetic Heisenberg coupling JH > 0 in unitsof the bare hopping amplitude t0, while the doping level δ is given onthe vertical axis. The color code describes the magnitude of the criticalscale $c in units of the bare hopping t0 across the phase diagram. Thedashed black line marks the transition to a topological odd-paritypairing state across the van Hove singularity. In this approximation,we obtain several superconducting instabilities, where labels referto intraband pairing symmetries. See the main text for a detaileddescription.

of our present results. A projection of the initial conditiononto pair-scattering amplitudes in the singlet regime revealsa d-wave dominance for δ < 0.4 and a subleading s-wave,while for δ > 0.4, the situation is reversed and the s-wave formfactor is dominating. The form factor with the largest weight issubsequently enhanced by the flow. We also performed flows atfinite temperature, which, however, showed that temperaturedoes not exert an influence on the respective d- or s-wavedominance in our flows. Rather, above the critical temperaturethe singular flow is smoothed, which signals the stability of aFermi liquid ground state.

Due to lattice symmetry, the intraband d-wave solution isdoubly degenerate, i.e., the largest eigenvalue of the singletpair scattering comes with a two-dimensional eigenspace.Projecting onto the form factors given in Appendix A2, wefind that each eigenvector has overlap with the two even-paritynearest-neighbor d-wave form factors dxy(k) and dx2−y2 (k) asdefined by Eqs. (A6) and (A5). See Fig. 8 for the momentum-space structure of the divergent singlet vertex function and thecorresponding eigenvectors of the pair-scattering amplitudes.We note that due to the lack of particle-hole fluctuations inthis reduced flow, no longer-ranged intrasublattice pairingcorrelations develop.

The interband pair scattering shows odd-parity p-wavecorrelations in the singlet regime, also with degenerate p-waveform factors px(k) and py(k) on nearest-neighbor bonds asdefined by Eqs. (A7) and (A8). The interband correlations are,in fact, substantial close to $c and comparable in magnitude

5 10 15 200.4

0.2

0.0

0.2

0.4

patch k

vk ,Λ

5 10 15 200.4

0.2

0.0

0.2

0.4

patch k

vk ,Λ

FIG. 8. (Color online) The upper left panel shows the intrabandcomponent of the divergent singlet vertex function for δ = 0.2and JH/t0 = 0.9, where the patch numbers corresponding to patchmomentum k1 are given on the ordinate and k2 on the abscissa.The remaining free momentum k3 is fixed to the first patch, cf.Fig. 6. The divergent momentum structure corresponds to a d-waveinstability. The upper right panel shows the intraband component ofthe divergent singlet vertex function for δ = 0.44 and JH/t0 = 1.31,with a momentum structure yielding an s-wave instability. In thelower left panel, we display the normalized amplitude of the twodegenerate eigenvectors with d-wave symmetry of the singlet pair-scattering amplitude along the Fermi surface. The patch numberis enumerated on the abscissa. The lower right panel shows thecorresponding eigenvector with s-wave symmetry.

045135-11


to intraband correlations. On the level of our fRG-flows thisremains unchanged when turning on finite temperature.

At large doping δ > 0.4, the leading intraband correlationschange from d to s wave with even-parity nearest-neighborform factor, see Eq. (A4), corresponding to an extended s-wavepairing instability.

The degeneracies in the case of singlet instabilities arerelated to lattice symmetries [57,66,67]. Which linear com-bination is finally realized in the superconducting state needsto be inferred from, e.g., a comparison of ground-state energies[68] as obtained from mean-field theory. Only when self-energy feedback or counter terms [61,69,70] are included inthe fRG flow, symmetry breaking can be accounted for.

The triplet instability for JH " |JK|/2 is manifested bydiverging triplet vertex functions. As noted in Sec. IV, thediscrete symmetry of the Kitaev interaction relates the tripletfunctions among each other. Since the flow stays in thesymmetric regime, the triplet vertex functions diverge simul-taneously. Moreover, symmetry ensures that the eigenvaluesobtained from diagonalizing the triplet pair scattering are alsodegenerate. From the current fRG scheme, we can thus inferthree degenerate d vectors, each one corresponding to one ofthe degenerate triplet channels. As in the singlet case, the trueground state will pick a particular linear combination, whichis, however, inaccessible in the employed scheme. Since theWard identity (see Sec. A4) derived from the discrete Kitaevsymmetry allows reconstruction of two vertex functions froma given one, we only keep the triplet vertex V (t)

x in the flow.The vertices V (t)

y and V (t)z are obtained from the final result

for V (t)x .

The triplet instability is dominated by intraband pairing,which in fact corresponds to the p-SC solution found inRef. [33]. From an analysis of the pair-scattering amplitude, weobtain with a high numerical accuracy the degenerate solutions(see also Fig. 9)

dk,1 = [px(k) −√

3 py(k)](1,0,0)T ,

dk,2 = [px(k) +√

3 py(k)](0,1,0)T , (27)

dk,3 = [2 px(k)](0,0,1)T .

5 10 15 200.4

0.2

0.0

0.2

0.4

patch k

vk ,Λ

FIG. 9. (Color online) The left panel shows the intraband com-ponent of the divergent triplet vertex function for δ = 0.3 andJH/t0 = 0.375, where the patch numbers corresponding to patchmomentum k1 are given on the ordinate and k2 on the abscissa. Theremaining free momentum k3 is fixed to the first patch, cf. Fig. 6. Thedivergent momentum structure corresponds to a p-wave instability.In the right panel, we display the normalized amplitude of the p-wave eigenvectors [px(k) −

√3 py(k)] of the triplet-x pair-scattering

amplitude along the Fermi surface. The patch number is enumeratedon the abscissa.

Expanding these form factors to leading order in k aboutthe % point in the BZ, we recover the results obtained inRef. [33]. There it was also shown that such a d-vectorconfiguration realizes pairing between fermions with spinprojections aligned [with ∼(kx − iky) pairing] or antialigned[with ∼(kx + iky) pairing] along the (1,1,1)T axis in spinspace. Further following mean-field arguments [33] andemploying knowledge about the mechanism for the creationof topological pairing, namely an odd number of time-reversalinvariant points enclosed by the Fermi surface [10,11,37–39],for δ > 0.25 (indicated by the black dashed line in Fig. 7)the triplet p-wave superconductor turns into a topologicalsuperconductor.

In total, we obtain good agreement with results from mean-field theory [33,67] from the reduced pure particle-particleflows. Also when turning to the stability of the superconductingphases with respect to thermal excitations, we obtain estimatesfor critical temperatures from the critical scale $c (see Fig. 7)that are within the same orders of magnitude as reported inRef. [33]. Within the p-wave phase, the critical temperaturedecreases from kBTc ∼ 10−2 t0 at δ ! 0.1 by two orders ofmagnitude to kBTc ∼ 10−4t0 at δ ! 0.3. For fixed dopinglevel, the critical scale/temperature remains constant withinthe p-wave phase. This, however, can be easily understoodfrom the fixed Kitaev coupling JK/t0 = −1. Since singletand triplet vertices are decoupled in the reduced flows, thechannel that diverges first wins the race and determines theleading instability. The singlet vertex thus has no influenceon the critical scale within the p-wave regime determined bythe leading triplet channel, see also Fig. 10. For parametersin the singlet regime, the critical scale shows a prominentJH dependence. Increasing JH, the critical scale/temperaturegrows rather quickly to larger values kBTc ∼ 10−1 t0 forδ ! 0.1. A logarithmic plot of the critical scale as a functionof JH/t0 for various dopings is given in Fig. 10, where the

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.410 5

10 4

0.001

0.01

0.1

JH t0

ct 0

FIG. 10. (Color online) The critical scale $0 in units of the barehopping t0 as a function of the antiferromagnetic Heisenberg couplingJH, also in units of t0 and for fixed JK/t0 = −1. The plateaus ofconstant critical scales below critical Heisenberg coupling can beclearly identified. See main text for an explanation. $/t0 is heregiven on a logarithmic scale. Starting with doping δ = 0.1, thedoping level decreases from the top to the bottom curve in steps of1δ = 0.02. Blue corresponds to doping δ = 0.1, . . . ,0.24, orange toδ = 0.26, . . . ,0.38, and red to δ = 0.4, . . . ,0.5.

045135-12


plateaus for fixed doping within the triplet regime can beclearly identified.

2. Unbiased resummation of particle-particleand particle-hole bubbles

Having established our method in the limit of ex-clusive particle-particle contributions to the flow ofthe scale-dependent vertex functions, we now include theparticle-hole fluctuations. These lead to a coupling of singletand triplet vertex functions. The particle-hole contributionsare in fact considerably more complicated than the particle-particle contributions alone. This originates from our choiceof channel decomposition of the initial condition, cf. Eqs. (15)and (16).

The resulting phase diagram is presented in Fig. 3. The p-wave instability seems to be largely unaffected by the inclusionof particle-hole fluctuations. As in the pure particle-particlecase, symmetry guarantees degeneracy of the triplet vertices.We even find that the d vector describing the triplet instabilityis still rather well described by the form given in Eq. (27).Particle-hole fluctuations, however, generate longer-rangedpairing correlations. In the triplet channel, these are subleadingcontributions compared to the leading nearest-neighbor pwave. For intermediate JH and δ, the leading instability stilloccurs in the singlet channel with d-wave symmetry, and forlarger doping δ ! 0.4 the order-parameter symmetry switchesto s wave. The phase boundaries between the adjacent super-conducting instabilities appear to be rather robust with respectto particle-hole fluctuations as compared to the previous pureparticle-particle resummation. Critical scales and temperaturesare also only mildly affected. We plot the critical scalelogarithmically in Fig. 11 for various dopings as a functionof JH/t0. We do no longer find a constant $c for fixed dopingand JK/t0 = −1 as JH is varied within the p-wave tripletregime. As expected, the particle-hole fluctuations suppress thecritical scale in the superconducting regimes. Quantitatively,the changes as compared to the pure particle-particle case

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.410 5

10 4

0.001

0.01

0.1

JH t0

ct 0

FIG. 11. (Color online) The critical scale $0 in units of thebare hopping t0 as a function of the antiferromagnetic Heisenbergcoupling JH, also in units of t0 and for fixed JK/t0 = −1. $/t0 ishere given on a logarithmic scale. Starting with doping δ = 0.1, thedoping level decreases from the top to the bottom curve in steps of1δ = 0.02. Blue corresponds to doping δ = 0.1, . . . ,0.24, orange toδ = 0.26, . . . ,0.38, and red to δ = 0.4, . . . ,0.5.

FIG. 12. (Color online) The upper panel shows the intrasublattice(left) and intersublattice (right) component of the divergent singletvertex function for δ = 0.12 and JH/t0 = 0.75 with JK/t0 = −1,where the patch numbers corresponding to patch momentum k1 aregiven on the vertical and k2 on the horizontal axis. The remainingfree momentum k3 is here fixed to the second patch, cf. Fig. 6.With the same conventions, the lower panel shows the intrasublattice(left) and intersublattice (right) component of the divergent triplet-xvertex function. The divergent momentum structure corresponds to anantiferromagnetic Neel instability. Both amplitude ratio of singlet totriplet vertex and the sign structure conspire to recombine singlet andtriplet pairing interactions into a spin-spin interaction, cf. Eq. (28).

reach up to an order of magnitude, cf. Fig. 10. Finally, inthe large-JH regime, the character of the instability changesfrom superconducting to magnetic. This can be read off fromthe singlet and triplet vertex functions as shown in Fig. 12.In the case of spin or charge density wave (SDW, CDW)instabilities, the singlet and triplet vertex functions encodethe corresponding divergent momentum structure in a rathercomplicated way due to the channel decomposition that isadapted to pairing instabilities. Nevertheless, the form ofthe full vertex function V $ can in these cases be obtainedessentially by matrix algebra3 and the momentum structurescorresponding to SDW and CDW instabilities can be obtained.Using Fierz identities and recombining singlet and tripletpairing channels, we recover a Hamiltonian

HAF ∝ −V∑

o,o′∈A,B

ϵo,o′ Soq=0 · So′

q=0, V > 0, (28)

that describes the low-energy degrees of freedom close to thecritical scale $c. Here, So

q =∑

k f†o,k,σ

[σ ]σσ ′fo,k−q,σ ′ is the q

component of the fermion spin operator in sublattice o ∈ A,B.The prefactor ϵo,o′ equals −1 for o = o′ corresponding to an-tiferromagnetic correlations between the two sublattices. Foro = o′, ϵo,o′ = +1, which describes ferromagnetic correlationsin a given sublattice. The long-range order correspondingto such a Hamiltonian with infinitely ranged interaction isnothing but a two sublattice Neel state, i.e., a commensurate

3The actual computations are most efficiently performed withthe help of so-called Fierz identities, which can be understoodas “rearrangement” formulas for the index structure of a quarticinteraction term.

045135-13


antiferromagnet, where the staggered magnetization isarranged over the two sublattices.

The momentum structure displayed in Fig. 12 is ratherbroad and smeared out. We confirmed that these features arealso obtained from a Hubbard model in the large-U regime onthe honeycomb lattice, where the Neel antiferromagnet wasestablished as the magnetically ordered ground state.

B. Doping QSL and zigzag phase: AF Kitaev and FMHeisenberg exchanges

To analyze the effect of doping charge carriers intothe QSL and the magnetically ordered zigzag phase, weselect the parameter range of the ferromagnetic (JH < 0)Heisenberg coupling as |JH|/t0 ∈ [0,1.5], while again keepingthe now antiferromagnetic Kitaev coupling fixed JK/t0 = 1.This parameter range covers both QSL and zigzag phase atδ = 0. The phase diagram extracted from fRG-flows with bothpartice-particle and particle-hole bubbles is shown in Fig. 4.We again find singlet and triplet pairing instabilities, whereas in the case of doping the QSL/stripy phase, the singletinstability comes with different pairing symmetries dependingon the doping level. Here, we find two different density-waveregimes (SDW, CDW). A further type of density wave state, abond-order wave, occurs at the special filling δ = 1/4, i.e., vanHove filling. This type of instability will be discussed below inSec. V C after we presented our findings for superconductingand SDW/CDW regimes.

At small doping δ ≃ 0.1 and for |JH|/t0 ≃ 0.1, the leadinginstability is of SDW type. In fact, the divergent momentumstructure is the same in the doped stripy phase, cf. Fig. 12. Wethus find a Neel antiferromagnet in this parameter range drivenhere by the antiferromagnetic Kitaev exchange. As both δ and|JH| are increased, the magnetic order rather quickly makesway for pairing instabilities and an adjacent CDW instability.The vertex structure corresponding to a CDW instability isshown in Fig. 13.

The charge density wave is produced by particle-holefluctuations, in a similar fashion as the antiferromagneticinstability. A pure particle-particle flow would of courseyield a superconducting instability, while a pure particle-holeresummation already gives us the CDW instability. The originof the strong CDW ordering tendencies traces back to therepulsive (for JH > 0) nearest-neighbor interaction betweenthe sublattice charge densities, see Eq. (4). While we thusfind the resulting phase diagram as a “competition” of tenden-cies, the existence of either one of the instabilities does nothinge on an interplay between different, competing channels.Such behavior would manifest itself in the complete absence ofa particular instability once either particle-particle or particle-hole bubbles are excluded from the flow. This observationcan be traced back to the t-JK-JH model that we take as ourstarting point. Since important particle-hole fluctuations of amicroscopic model in the Mott insulating phase are alreadycontained in the exchange terms, the subsequent fRG-flowtends to enhance the “preformed” tendencies. Similar to thecase of the antiferromagnet, the low-energy degrees of freedomclose to $c can be described by a Hamiltonian of the form

HCDW ∝ −V∑

o,o′∈A,B

ϵo,o′ Noq=0 No′

q=0, V > 0. (29)

FIG. 13. (Color online) The upper panel shows the intrasublattice(left) and intersublattice (right) component of the divergent singletvertex function for δ = 0.14 and |JH|/t0 = 0.525 with JK/t0 = 1,where the patch numbers corresponding to patch momentum k1 aregiven on the ordinate and k2 on the abscissa. The remaining freemomentum k3 is here fixed to the second patch, cf. Fig. 6. With thesame conventions, the lower panel shows the intrasublattice (left)and intersublattice (right) component of the divergent triplet-x vertexfunction. The divergent momentum structure corresponds to a CDWinstability. Both amplitude ratio of singlet to triplet vertex and the signstructure conspire to recombine singlet and triplet pairing interactionsinto a density-density interaction, cf. Eq. (29).

Here, Noq=0 =

∑k f

†o,k,σ

[σ0]σσ ′fo,k−q,σ ′ is the sublattice den-sity operator for auxiliary fermions. It differs from the electrondensity only by a factor of δ. The system minimizes itsenergy by having a charge imbalance, e.g., more electronsreside on sublattice A than on sublattice B, or vice versa.The CDW instability takes up a large part of the phasediagram and also comes with rather large critical scales. Weestimate critical temperatures up to kBTc ∼ 10−1 t0. Previousmean-field studies did not include CDW order-parameters. Inour case, the CDW instability is driven by the density-densityterm in Eq. (4) for JH < 0 and outweighs ferromagneticordering tendencies.

We now turn to the superconducting instabilities. Thesinglet channel determines the leading superconducting in-stability only in a rather narrow strip for |JH|/t0 < 0.2. Fordoping up to δ = 0.4, the intraband pairing symmetry isd-wave. Interband correlations are again of p-wave type.As the doping level is increased above δ = 0.4, an s-wavepairing symmetry is favored. Also here, the dominatingsuperconducting correlations are of nearest-neighbor type.Intrasublattice correlations are subleading.

As the strength of the ferromagnetic Heisenberg couplingis increased, at |JH|/t0 ≃ 0.2 the leading instability switchesfrom singlet to triplet. The different ordering tendencies inparticle-hole and particle-particle channels lead to a suppres-sion of intraband-pairing. On moving closer to the triplet-SC/CDW phase boundary, the intraband pairing correlations ofp-wave type grow stronger (see Fig. 14). Pairing correlationsalong nearest-neighbor bonds still dominate. For the intrabandcorrelations, however, we still observe a substantial decreaseas compared to ferromagnetic Kitaev and antiferromagneticHeisenberg exchange (cf. Secs. V A 1 and V A 2), while the

045135-14


5 10 15 200.4

0.2

0.0

0.2

0.4

patch k

vk ,Λ

5 10 15 200.4

0.2

0.0

0.2

0.4

patch k

vk ,Λ

FIG. 14. (Color online) The upper left panel shows the divergentpart of the interband triplet-x vertex function corresponding to d-wavepairing symmetry for δ = 0.34 and |JH|/t0 = 0.525 with JK/t0 = 1,where the patch numbers corresponding to patch momentum k1 aregiven on the ordinate and k2 on the abscissa. The remaining freemomentum k3 is fixed to the third patch for better visibility, cf. Fig. 6.The upper right panel shows the effect of the proximity to the CDWinstability in the intraband triplet-x vertex function. The correlationsare of nearest-neighbor p-wave type. The lower left panel shows theeigenvector with mixed s- and d-wave symmetry of the interbandtriplet-x pair-scattering amplitude along the Fermi surface. The patchnumber is enumerated on the abscissa. The lower right panel showsthe corresponding intraband eigenvector with p-wave symmetry.

interband correlations dominate. The intraband pairing can bedescribed by the following d vector:

dk,1 = [px(k) + 1/√

3 py(k)](1,0,0)T ,

dk,2 = [px(k) − 1/√

3 py(k)](0,1,0)T , (30)

dk,3 = [2/√

3 py(k)](0,0,1)T .

As compared to the d vector obtained from doping thestripy phase, here the p-wave instability is driven by theferromagnetic and isotropic Heisenberg exchange.

The interband d vector is captured by (see also Fig. 14)

dk,1 = [−s(k) + 1/3 dxy(k) + 1/3√

3 dx2−y2 (k)](1,0,0)T ,

dk,2 = [s(k) + 1/3 dxy(k) − 1/3√

3 dx2−y2 (k)](0,1,0)T , (31)

dk,3 = [s(k) + 2/3√

3dx2−y2 (k)](0,0,1)T .

Here, by s(k) we denote the even parity nearest-neighbor s-wave form factor, see Eq. (A4). The p-wave part in Eq. (30)was reported previously [35] with dominant intraband pairing.We here find dominating interband correlations and enhanceds-wave contributions close to the CDW phase boundary.

Critical scales/temperatures for the p-wave regime decreaseupon doping from kBTc ∼ 10−4 t0 to kBTc < 10−8 t0. Criticalscales below 10−8 t0 could actually not be properly resolvedfrom the fRG-flows, see also Fig. 15. Further, critical scalesare not constant along the JH axis for fixed δ and JK.

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.410 9

10 7

10 5

0.001

0.1

JH t0

ct 0

FIG. 15. (Color online) The critical scale $0 in units of the barehopping t0 as a function of the ferromagnetic Heisenberg couplingstrength |JH|, also in units of t0 and for fixed JK/t0 = 1. $/t0 ishere given on a logarithmic scale. Starting with doping δ = 0.1, thedoping level decreases from the top to the bottom curve in steps of1δ = 0.02. Blue corresponds to doping δ = 0.1, . . . ,0.24, orange toδ = 0.26, . . . ,0.38, and red to δ = 0.4, . . . ,0.5. For scales $/t0 <

10−8, marked by the solid blue line, the fRG flow could not beevaluated properly to even lower scales for the given choice ofparameters used for numerical integration of the flow equation.

C. Bond-order instabilities at van Hove filling

The filling δ = 1/4 plays a special role in honeycomb latticemodels, since perfect Fermi surface nesting and a van Hovesingularity coincide. It is thus not surprising, that the effectsfrom nesting and enhanced density of states (DOS) at theFermi level lead to a strong impact from the particle-holefluctuations on the emerging Fermi surface instability. Since itis the interplay of nesting and density-of-states enhancementthat is important, the ensuing phase at van Hove filling shouldbe considered as rather fragile with respect to deviations infilling factor. It comes, however, with an increased criticalscale, i.e., larger critical temperature due to larger Fermi levelDOS, cf. Fig. 16. Additionally, we find that for the parameterranges studied in this work, only for antiferromagnetic Kitaevand ferromagnetic Heisenberg exchange do the particle-holeeffects outweigh the pairing instability. Consequently, we willfocus on JK > 0 and JH < 0 in the following. As before,we keep the Kitaev interaction fixed at JK/t0 = 1 and varythe Heisenberg exchange |JH|/t0 ∈ [0,1] for fixed dopingδ = 1/4. It turns out that an N = 24 patching scheme isinsufficient to properly capture both DOS enhancement andnesting, and leads to spurious artefact instabilities throughoutthe phase diagram. Upon increased angular resolution alongthe Fermi surface, these artefacts disappear at N = 96 andallow for a clear identification of the resulting orderingstructures. Since N = 24 patching has proven quite reliablein interacting honeycomb systems away from van Hove filling[71–73], we believe our results for δ = 1/4 are quite robust.We supported this claim by checking the phase boundaries inFig. 4 with N = 96. Only the singlet/triplet phase boundarywas mildly affected.

The nesting vectors Qi , i = 1,2,3 connect opposite edgesof the hexagonal Fermi surface at van Hove filling, seeFig. 6. Modulo reciprocal lattice vectors, these are equivalent

045135-15


0.0 0.2 0.4 0.6 0.8 1.010 11

10 9

10 7

10 5

0.001

0.1

JH t0

ct 0

FIG. 16. (Color online) Critical scale $c in units of the barehopping t0 for flows evaluated at van Hove filling δ = 1/4 (magenta)and fillings δ = 1/4 − 0.01 (blue) and δ = 1/4 + 0.01 (orange) asa function of ferromagnetic Heisenberg exchange |JH|/t0 and fixedantiferromagnetic Kitaev exchange JK/t0 = 1. As expected from theenhancement of the single-particle density of states for the nestedFermi surface at van Hove filling, the critical scale is enhanced bya few orders of magnitude. It drops, however, rather quickly as thedoping departs from δ = 1/4. Also, the effect of DOS enhancementreflected in increased critical scales is rendered ineffective as soon asthe CDW instability sets in at |JH|/t0 ! 0.7.

to vectors connecting inequivalent, neighboring M points.Explicitly, they are given by

Q1 = π (1/√

3,1)T , Q2 = π (−1/√

3,1)T ,

Q3 = π (−2/√

3,0)T . (32)

An emergent order parameter with ordering wave vector Qi

breaks translation invariance of the underlying lattice andleads to a doubling of the unit cell, i.e., a four-atom unitcell in our present case. From analyzing the momentum-spacepattern of the renormalized vertex function at the critical scaleand employing Fierz identities, we find effective low-energyHamiltonians of either charge bond-order (cBO) or spinbond-order (sBO) type:

HcBO ∝3∑

i=1

V icBO2∗

Qi2Qi

, (33)

HsBO ∝3∑

i=1

∑

l∈{x,y,z}V i,l

sBO2∗l,Qi

2l,Qi, (34)

with charge and spin bond-order amplitudes V icBO and V i,l

sBO,respectively. The fermion bilinears 2Q and 2l,Q are given by

2Q ∝∑

k,σ

∑

o,o′

ϵo,o′ to,o′

k(Q)f †

o,k,σfo′,k−Q,σ (35)

and

2l,Q ∝∑

k,σ,σ ′

∑

o,o′

ϵo,o′ to,o′

l,k(Q)f †

o,k,σ[σl]σσ ′fo′,k−Q,σ ′ , (36)

where i = 1, . . . ,3 labels the different ordering wave vec-tors, and l ∈ {x,y,z} labels spin-vector components. Here,

FIG. 17. (Color online) Real-space charge bond-order patternsfor |JH|/t0 = 0.2. Red bonds correspond to enhancement of thehopping amplitude along the given bond, while blue bonds correspondto a decrease of the hopping amplitude. White disks mark the A

sublattice sites, black disks the B sublattice sites. Shown are theordering patterns for the three different ordering wave vectors Q1

(left), Q2 (middle), and Q3 (right). The form factors are given bycos(δ3 · k), cos(δ1 · k), and cos(δ3 · k), respectively.

ϵo,o′ = +1 for o = o′ and ϵo,o′ = 0 for o = o′. This par-ticular form of interlattice correlations corresponds to thedimerization of particle-hole excitations along a given bond.On a mean-field level, a finite expectation value ⟨2Q⟩of the cBO order-parameter leads to a renormalization ofthe hopping amplitude and to an enlargement of the unitcell with a corresponding downfolded Brillouin zone andadditional bands. From numerical calculations, we find theform factors to,o′

k(Q) can be described by cos(δj · k), j = 1,2,3

form factors for hopping along nearest-neighbor bonds. Theresulting real-space patterns are displayed in Fig. 17. A finitesBO order parameter ⟨2l,Q⟩ leads to a renormalized spin-dependent hopping amplitude. For given Q, the form factorsto,o′

k(Q) and to,o′

l,k(Q), respectively, determine the bond-order

pattern within the enlarged unit cell. From our fRG resultswe infer the leading instability is either of cBO or sBOtype, but the two different instabilities do not coincide. Theeigenmodes for different Q extracted from the correspondingreduced vertex functions—which can again be understoodas N×N matrices—turn out to be degenerate. Further, inthe sBO case, there are always two (almost) degenerateeigenmodes with different l for fixed Q. The association ofspin matrices to a given ordering wave vector as obtainedfrom our numerical results is collected in Table I. For sBOinstabilities, the dominant features of the numerically obtainedform factors can be described with sin(δj · k), j = 1,2,3 whereδj are the nearest-neighbor vectors from A to B sublattice,see Fig. 18. These, of course, can be expressed in termsof nearest-neighbor p-wave form factors. The form factors,however, seem to rotate in the degenerate p-wave subspace

TABLE I. Spin matrices associated with the three differentordering wave vectors as obtained from our numerical results. Forfixed wave vector Q, emergent spin bond order yields spin-dependenthopping described by the checked spin matrices within a mean-fieldtreatment of the low-energy Hamiltonian (34).

Wave vector σx σy σz

Q1 # – #Q2 – # #Q3 # # –

045135-16


FIG. 18. (Color online) Real-space spin bond-order patterns for|JH|/t0 = 0.3. Red bonds correspond to enhancement of the hoppingamplitude along the given bond, while blue bonds correspond to adecrease of the hopping amplitude. White disks mark the A-sublatticesites, black disks the B-sublattice sites. Shown are the orderingpatterns for the three different ordering wave vectors Q1 (left),Q2 (middle), and Q3 (right). The spin-dependent form factors(see Table I) are given by σz sin(δ3 ·k), σy sin(δ3 ·k), and σx sin(δ1 ·k),respectively.

as JH changes. The modes corresponding to cos(δj · k) formfactors turn out to be subleading for the sBO instability.Fourier transforming the form-factors yields the correspondingmodulation of the real-space hopping amplitude. Due to thelimitations of our truncation to the exact hierarchy of fRGequations, we cannot determine which linear combinationof the different mean-fields will be realized in the groundstate of the system. While some of the sine patterns overlap,others reside on mutually exclusive bonds. For overlappingpatterns, we cannot expect the different ordering patterns tobe energetically independent. A determination of the lowest-energy configuration, however, is beyond the capabilities ofour employed truncation scheme.

As displayed in Fig. 4, at small |JH|/t0, the singlet pairinginstability is leading, while at |JH|/t0 = 0.2, a charge bondorder sets in as the leading instability. As |JH|/t0 increases,the leading instability quickly crosses over from charge to spinbond order with the aforementioned two degenerate eigen-modes per ordering wave vector. The bond-order instability iscut off by the CDW for |JH|/t0 ! 0.7

In view of the superconducting neighborhood of the bond-order instabilities at van Hove filling, cf. Fig. 4, we can inferthat while the proximity to even-parity singlet pairing alsopromotes even-parity singlet charge bond order, odd-paritytriplet pairing favors the formation of odd-parity triplet spinbond order.

Remarkably, even though we modeled the hopping in thenoninteracting Hamiltonian (2) as spin-independent, the inter-play of antiferromagnetic Kitaev and ferromagnetic Heisen-berg exchanges with nesting and DOS enhancement leadto dynamical regeneration of anisotropic spin-orbit couplingtype terms on the level of a mean-field treatment of thelow-energy Hamiltonian (34). While a detailed analysis ofthe properties of fermions moving in the background ofself-consistently generated bond-order patterns is beyond thescope of this paper, the mean-field Hamiltonian for low-energyfermions with a static bond-order mean-field readily yieldsa renormalized fermion spectrum. Considering the differentordering wave vectors independently, we obtain a metallicstate for each Q with a connected Fermi surface in the reducedBrillouin zone. Energetically, a gapless metallic state mightseem less favorable for the system than a state with a nodal

superconducting gap. But the condensation of bond orderseems to occur at critical scales which are well above thecritical temperature for the transition to the superconductingstate with a nodal gap along the Fermi surface.

We note that even for JK = 0, a finite ferromagneticHeisenberg coupling JH is sufficient to drive the systemtoward a bond-order instability at van Hove filling. A similarobservation—ferromagnetic fluctuations causing a propensitytoward bond-order instabilities—was made for the extendedkagome Hubbard model with fRG methods [52]. The dimer-ization pattern corresponds to spin bond order, due to therestored rotational symmetry, however, all spin componentsare degenerate. We attribute the fact that we do not observe amagnetically site-ordered state to the dominating role of thenearest-neighbor density-density interaction term, cf. Eq. (4).For δ = 1/4 and JK = 0, critical scales drop quickly below10−8t0. The associated instabilities, if they exist, are thus notobservable within our current approach.

Finally, we comment on the stability of our results inthe presence of Fermi surface renormalization. Since inthe present truncation self-energy feedback is completelyneglected, the shape of the Fermi surface is fixed during theRG flow. An fRG scheme that takes into account self-energyfeedback in principle can modify the Fermi surface and thusmight destroy the nesting condition. However, as observedin Ref. [74] the real part of the flowing self-energy mainlyleads to a straightening of the Fermi surface. From thispoint of view, it seems plausible that effects from nestingand DOS enhancement are stable with respect to inclusionof self-energy effects. A more complete picture studying theinterplay between van Hove singularities and self-energy flowis, however, certainly desirable.

VI. CONCLUSIONS AND DISCUSSION

We have analyzed the phase diagram of the doped Kitaev-Heisenberg model on the honeycomb lattice for the situationsof ferromagnetic Kitaev and antiferromagnetic Heisenbergexchanges, as well as antiferromagnetic Kitaev and ferro-magnetic Heisenberg exchanges. We attacked the problem ofdescribing the correlated, frustrated, and spin-orbit coupledMott insulator within a slave-boson treatment, and derivedfunctional RG equations for the auxiliary fermionic problemafter the bosonic holon sector was dealt with on a mean-fieldlevel. We solved the functional flow equations in the staticpatching approximation, where the patch number for angularresolution of the Fermi surface ranged from N = 24 to 96.

While our results corroborate the tendency towards theformation of triplet p-wave pairing phases, we demonstratethat other competing orders driven by particle-hole fluctuationsreduce the parameter space where pairing yields the leadinginstability. We further uncovered instabilities at van Hovefilling supporting unconventional dimerization phases of theelectronic liquid. Interestingly, the prediction of emergenttopological p-wave pairing states is unaffected by the inclu-sion of particle-hole fluctuations. For ferromagnetic Kitaevand antiferromagnetic Heisenberg exchange, the gap-closingtransition from trivial to topologically nontrivial p wave isleft untouched, although critical temperatures are reduced.

045135-17


Flipping the signs of both exchange terms, a bond-orderinstability pre-empts the naive pairing mean-field gap-closingtransition at van Hove filling. The resulting dimerization state,however, remains gapless. Upon doping beyond van Hovefilling, the p-wave phase is restored. Applying the rule ofcounting the number of time-reversal invariant momenta belowthe Fermi surface [10,11,37–39], we again obtain a topologicalp-wave state.

While the dimerized state at van Hove filling appears toremain gapless and nontopological, the proposal of Ref. [32] toinclude longer-ranged exchange interactions beyond isospinsconnected by nearest-neighbor bonds to better model themagnetic state of Na2IrO3 might also provide a route todynamically generated topological Mott insulating states atvan Hove filling.

Extending the t-JK-JH model to a JH-V model in theHeisenberg sector, where V is now promoted to an independentcoupling for nearest-neighbor density-density interactions(while we restricted our attention to V = JH) provides anotherroute to generalization. At least for a subset of initial valuesfor JH and V , however, the fRG-flow will be attracted tothe infrared manifold and the corresponding instabilities wediscuss in the present paper. Further, the fRG approach takenin this work might be successfully applied to doping inducedinstabilities in the context of other material-inspired spin-orbitmodel Hamiltonians [75].

Before closing the discussion of our results, we brieflycomment on the treatment of the t-JK-JH model within the fRGframework. The fRG approach employed in this work differsfrom fRG applications to other, weakly correlated electronlattice systems (for a review, see Ref. [42]) as its startingpoint is the renormalized auxiliary fermion Hamiltonian,with a strongly reduced bandwidth. This may cast somedoubts on the applicability of a method perturbative in theinteractions like the fRG. Here, we do not claim that the resultsare quantitatively controlled, but we can be confident thatqualitatively they capture the right trends. First of all it shouldbe noted that using fRG instead of the common mean-fieldstudy of the phase diagram of the auxiliary fermion modelis certainly an improvement that removes ambiguities andexcludes that important channels may get overlooked. Thenwe also refer to a number of works with a method dubbed “spinfRG,” cf. Refs. [26,29,62]. In these works, related spin physicsis explored in the insulating limit where the kinetic energy iscompletely quenched. The results obtained there are physicallymeaningful and give insights into spin physics of frustratedmodels that are otherwise hard to obtain. In the insulatingcase, the fermion propagator is purely local and the spin-spininteraction remains of a simpler bilocal form. Hence its full fre-quency dependence can be taken into account. This simplicityis lost in the doped case studied here, as the fermion propagatoris nonlocal and mediates effective interactions different fromsimple bilocal spin-spin type. Hence, for us it is difficult totreat the frequency dependence of the vertex in addition to theeven more important momentum space structure. Nevertheless,as our case interpolates between the two extreme cases,insulator and weakly correlated systems, where the approachhas been shown to work reasonably, we can be confident thatstudying the correlated doped case by perturbative fRG isjustified.

ACKNOWLEDGMENTS

D.D.S. acknowledges discussions with L. Kimme, T. Hyart,and M. Horsdal and technical support by M. Treffkorn and H.Nagel. M.M.S. is supported by the grant ERC-AdG-290623.

APPENDIX: TECHNICAL SUPPLEMENT

1. Gamma matrices and superconducting order parameters

The decomposition of the interaction terms in the singletand triplet pairing channels is adapted to analyzing super-conducting instabilities. Accordingly, the % matrices werechosen following the conventions [57] used in the descriptionof unconventional superconductors. This way, the spin struc-ture of an emerging superconducting instability is includedautomatically in our approach. For convenience, we here givethe explicit expressions for the complete set of 2×2 matrices:

%0 = 1√2σ0iσy = 1√

2

(0 1

−1 0

),

%x = 1√2σx iσy = 1√

2

(−1 00 1

),

%y = 1√2σy iσy = 1√

2

(i 00 i

),

%z = 1√2σziσy = 1√

2

(0 11 0

), (A1)

Tr(%µ%†ν) = δµν, µ,ν ∈ {0,x,y,z}. (A2)

Here, σx , σy , and σz are the Pauli matrices, and σ0 denotesthe 2×2 unit matrix. Further, one can derive so-called Fierzidentities for this set of matrices in order to rewrite quarticpairing terms into density-density type interactions. Thisenables us to obtain CDW and SDW type instabilities fromthe singlet and triplet pairing interactions.

The most general superconducting order parameter withsinglet and triplet components can now be compactly writtenas [57]

1k =√

2(ψk %0 + dk · %), (A3)

with % = (%x,%y,%z)T . The singlet order parameter isdescribed by a scalar function ψk , while the tripletorder parameter is specified by a three-component vectordk = (dk,x,dk,y,dk,z)

T . Since in this work, we are dealing witha multiband system, the 2×2 order-parameter 1k also carriesband indices describing intra- ([ψk]b1,b2 , [dk]b1,b2 with b1 = b2)and interband pairing ([ψk]b1,b2 , [dk]b1,b2 with b1 = b2).

2. Form factors on the honeycomb lattice

The form factors that we employ here for the analysis oforder-parameter symmetries are obtained from the irreduciblerepresentations of the point group of the hexagonal lattice.For a given representation in terms of functions defined onthe real-space lattice, a momentum space form factor fk canbe obtained as fk =

∑r eik·r fr . As is usually done, the lattice

sum is split into nearest-neighbor (NN), next-nearest neighbor(NNN), etc., contributions. The NN form factors are thus givenby fk =

∑j eik·δj fr+δj

. Since the resulting form factors do not

045135-18


come with well-defined parity, we form the appropriate linearcombinations yielding form-factors that are either even or oddwith respect to k → −k. We can proceed accordingly for NNNform factors and so on. The obtained set of form factors issuitable to analyze order-parameter symmetries for fermionsin Bloch/sublattice representation. When we switch to the bandrepresentation, the NN form factors pick up a phase factorφk =

∑j eik·δj /|

∑j eik·δj | due to the unitary transformation

relating Bloch and band representations. For the NN formfactors in the band representation, we find the followingexpressions:

s(k) = 13

[cos

(√3kx

2− ky

2+ φk

)

+ cos(√

32

kx + ky

2− φk

)+ cos(ky + φk)

],

(A4)

dx2−y2 (k) = −43

[cos

(√3kx

2− ky

2+ φk

)

+ cos(√

32

kx + ky

2− φk

)− 2 cos(ky + φk)

],

(A5)

dxy(k) =8 sin

(√3kx

2

)sin

( ky

2 − φk

)√

3, (A6)

px(k) =2 sin

(√3kx

2

)cos

( ky

2 − φk

)√

3, (A7)

py(k) = 13

[− sin

(√3kx

2− ky

2+ φk

)

+ sin(√

32

kx + ky

2− φk

)+ 2 sin(ky + φk)

].

(A8)

In the limit φk → 0, we recover the NN form factors in theBloch/sublattice representation.

3. Flow equations

In this section, we summarize the RG contributions to theright-hand sides of Eqs. (21) and (22), respectively. We herestick to the conventions of Ref. [60]. We define the shorthand∫dη to represent integration/summation over loop variables.

The spin projection σ = ↑,↓ is not included and has alreadybeen traced over in going from ξ to ξ and η to η. Carrying outthe projections onto singlet and triplet channels and definingthe loop kernel L = S$G$

0 + G$0 S$ with the single-scale

propagator S$ = d/d$ G$0 , we find for the singlet case the

particle-particle contribution

φ(s)pp (ξ1,ξ2,ξ3,ξ4) = 1

2

4∏

ν=1

∫dην L(η2,η1,η3,η4)

×V (s)(ξ2,ξ1,η2,η3)V (s)(η4,η1,ξ3,ξ4).

(A9)

The particle-particle bubble-contribution to the triplet channelis given by

φ(t)pp;l(ξ1,ξ2,ξ3,ξ4) = 1

2

4∏

ν=1


×V(t)l (ξ2,ξ1,η2,η3)V (t)

l (η4,η1,ξ3,ξ4).

(A10)

Obviously, particle-particle fluctuations do not couple sin-glet and triplet vertex functions. The singlet particle-holefluctuations read

φ(s)ph (ξ1,ξ2,ξ3,ξ4)

= −14

4∏

ν=1

∫dην L(η1,η2η3,η4)

×[

V (s)(η4,ξ2,ξ3,η1)V (s)(ξ1,η2,η3,ξ4)

+∑

i,j

V(t)i (η4,ξ2,ξ3,η1)V (t)

j (ξ1,η2,η3,ξ4)

+∑

i

V (s)(η4,ξ2,ξ3,η1)V (t)i (ξ1,η2,η3,ξ4)

+∑

i

V(t)i (η4,ξ2,ξ3,η1)V (s)(ξ1,η2,η3,ξ4)

]

, (A11)

while the triplet contribution is given by

φ(t)ph;l(ξ1,ξ2,ξ3,ξ4)

= −14

4∏

ν=1


×[

V (s)(η4,ξ2,ξ3,η1)V (s)(ξ1,η2,η3,ξ4)

+∑

i,j

clij V

(t)i (η4,ξ2,ξ3,η1)V (t)

j (ξ1,η2,η3,ξ4)

+∑

i

cil V(s)(η4,ξ2,ξ3,η1)V (t)

i (ξ1,η2,η3,ξ4)

+∑

i

cil V(t)i (η4,ξ2,ξ3,η1)V (s)(ξ1,η2,η3,ξ4)

]

. (A12)

The four coefficient matrices cil and clij that result from

performing internal spin summations encode a specific signstructure,

cil =

⎛

⎝+1 −1 −1−1 +1 −1−1 −1 +1

⎞

⎠

il

, cxij =

⎛

⎝+1 +1 +1+1 +1 −1+1 −1 +1

⎞

⎠

ij

cyij =

⎛

⎝+1 +1 −1+1 +1 +1−1 +1 +1

⎞

⎠

ij

, czij =

⎛

⎝+1 −1 +1−1 +1 +1+1 +1 +1

⎞

⎠

ij

,

(A13)

045135-19


and i,l ∈ {x,y,z}. The crossed and direct particle-hole contri-butions entering the flow equations (21) and (22) are defined asφ

(s)ph,cr(ξ1,ξ2,ξ3,ξ4) ≡ φ

(s)ph (ξ1,ξ2,ξ3,ξ4) and φ

(s)ph,d(ξ1,ξ2,ξ3,ξ4) ≡

φ(s)ph (ξ1,ξ2,ξ4,ξ3) for the singlet case. An analogous definition

holds for the triplet case.Further we note that a δ function taking care of global

momentum conservation can be factored out from the flowequations, which leaves only three independent momenta. Thefinal flow equations are formulated and implemented in termsof reduced vertex functions with three independent momenta.For the sake of convenience, we denote full and reduced vertexfunctions with the same symbol.

4. Vertex reconstruction from the Ward identity

The symmetries of the Hamiltonian (14) can be efficientlydescribed by embedding the two-dimensional honeycomb lat-tice into a three-dimensional cubic lattice [34]. Then a rotationaround the n = 1√

3(1,1,1)T axis by ±2π/3 corresponds to the

C3 or C−13 element, respectively, of the point group acting on

a site in the honeycomb lattice. This rotation also preservesthe sublattice index, i.e., both A and B sublattices are mappedonto themselves. We note that the coordinate system is adaptedto an embedding of the honeycomb lattice in a 3D cubic lattice[34]. Under a rotation by −2π/3, the spin components alongthe bonds are mapped as Sx → Sy , Sy → Sz, and Sz → Sx . Asexpected from the strong spin-orbit coupling scenario realizedin the iridates, the Kitaev term is only invariant under simul-taneous transformations of spin and lattice (orbital) degreesof freedom. The corresponding transformation on the latticeis a rotation Rn(θ ) with θ = 2π/3. This operation can also berepresented through combinations of reflections or rotationsand reflections. The two transformations (spin and latticerotation) taken together leave the Hamiltonian (14) invariant.The SU(2)-transformation matrix acting on the fermionicdegrees of freedom is given by Sn(−θ ) = exp(i/2 θ n · σ ).Point-group transformations acting on the real-space latticealso induce a representation on the Brillouin zone. BZmomenta accordingly transform as k → k′ = RT

n (θ )k.Moving to a functional formulation and replacing operator

valued fields (f †,f ) with Grassmann variables (f ,f ), thesymmetry of the system is expressed as the invariance of thegenerating functional. For the generating functional of 1PIvertices %[f ,f ], this statement reads as [76]

%[f ′,f ′] = %[f ,f ], (A14)

where the prime denotes transformed fields. We note that sincethe kinetic part of the Hamiltonian is also invariant under this

rotational symmetry, so are both the bare and the regularizedbare propagators in band representation. This in turn impliesthat the identity (A14) is valid also for the scale-dependentgenerating functional %$ for $ > 0. We thus obtain

%$[f ′,f ′] = %$[f ,f ]. (A15)

For the discrete rotational symmetry discussed above, wearrive at the following transformation rules for the fermionfields in momentum space:

fo,σ,k → f ′o,σ ′,k′ =

∑

σ,k

[Sn(θ )]σ ′σ δ(k′ − RT

n (θ )k)fo,σ,k ,

(A16)

fo,σ,k → f ′o,σ ′,k′ =

∑

σ,k

[S−1

n (θ )]σσ ′ δ

(k′ − RT

n (θ )k)fo,σ ′,k′ .

(A17)

Expanding both sides of Eq. (A15) in transformed andoriginal fields and using the explicit representation of thetransformation rules, we find the following set of the Wardidentities for the (scale-dependent) singlet and triplet vertexfunctions:

V (s)o1,o2,o3,o4

(k1,k2,k3,k4) = V (s)o1,o2,o3,o4

(k′1,k

′2,k

′3,k

′4), (A18)

V (t)x;o1,o2,o3,o4

(k1,k2,k3,k4) = V (t)y;o1,o2,o3,o4

(k′1,k

′2,k

′3,k

′4), (A19)

V (t)y;o1,o2,o3,o4

(k1,k2,k3,k4) = V (t)z;o1,o2,o3,o4

(k′1,k

′2,k

′3,k

′4), (A20)

V (t)z;o1,o2,o3,o4

(k1,k2,k3,k4) = V (t)x;o1,o2,o3,o4

(k′1,k

′2,k

′3,k

′4), (A21)

where we made the orbital indices explicit, i.e., V (s)o1,o2,o3,o4

(k1,k2,k3,k4) ≡ V (s)(ξ1,ξ2,ξ3,ξ4) and V(t)l;o1,o2,o3,o4

(k1,k2,k3,k4)≡ V

(t)l (ξ1,ξ2,ξ3,ξ4) with ξ = (ω,o,k) in the static approxi-

mation ω = 0. These Ward identities give us the importantinformation, that for one given triplet vertex function, the otherremaining two vertex functions can be reconstructed. This factwas exploited in designing an efficient numerical implemen-tation of the flow equations (21) and (22). The Ward identityfor the singlet vertex function was not directly employed inthe numerical implementation. Besides fermionic exchangesymmetry, however, it serves as an important consistencycheck for the numerical solution of the flow equation.

[1] K. v. Klitzing, G. Dorda, and M. Pepper, Phys. Rev. Lett. 45,494 (1980).

[2] R. B. Laughlin, Phys. Rev. B 23, 5632(R) (1981).[3] D. J. Thouless, M. Kohmoto, M. P. Nightingale, and M. den

Nijs, Phys. Rev. Lett. 49, 405 (1982).[4] Q. Niu, D. J. Thouless, and Y.-S. Wu, Phys. Rev. B 31, 3372

(1985).[5] M. Kohmoto, Ann. Phys. (NY) 160, 343 (1985).[6] B. A. Bernevig and S.-C. Zhang, Phys. Rev. Lett. 96, 106802

(2006).

[7] B. A. Bernevig, T. L. Hughes, and S.-C. Zhang, Science 314,1757 (2006).

[8] A. P. Schnyder, S. Ryu, A. Furusaki, and A. W. W. Ludwig,Phys. Rev. B 78, 195125 (2008).

[9] M. Z. Hasan and C. L. Kane, Rev. Mod. Phys. 82, 3045(2010).

[10] X.-L. Qi, T. L. Hughes, S. Raghu, and S.-C. Zhang, Phys. Rev.Lett. 102, 187001 (2009).

[11] X.-L. Qi, T. L. Hughes, and S.-C. Zhang, Phys. Rev. B 81,134508 (2010).

045135-20

http://dx.doi.org/10.1103/PhysRevLett.45.494
















http://dx.doi.org/10.1016/0003-4916(85)90148-4

http://dx.doi.org/10.1016/0003-4916(85)90148-4

http://dx.doi.org/10.1016/0003-4916(85)90148-4

http://dx.doi.org/10.1016/0003-4916(85)90148-4





http://dx.doi.org/10.1126/science.1133734








http://dx.doi.org/10.1103/RevModPhys.82.3045













[12] S. Ryu et al., New J. Phys. 12, 065010 (2010).[13] L. Balents, Nature (London) 464, 199 (2010).[14] X.-G. Wen and Q. Niu, Phys. Rev. B 41, 9377 (1990).[15] X.-G. Wen, Adv. Phys. 44, 405 (1995).[16] D. C. Tsui, H. L. Stormer, and A. C. Gossard, Phys. Rev. Lett.

48, 1559 (1982).[17] F. D. M. Haldane, Phys. Rev. Lett. 51, 605 (1983).[18] R. B. Laughlin, Phys. Rev. Lett. 50, 1395 (1983).[19] B. I. Halperin, Phys. Rev. Lett. 52, 1583 (1984).[20] A. Kitaev, Ann. Phys. (NY) 321, 2 (2006).[21] G. Jackeli and G. Khaliullin, Phys. Rev. Lett. 102, 017205

(2009).[22] J. Chaloupka, G. Jackeli, and G. Khaliullin, Phys. Rev. Lett.

105, 027204 (2010).[23] Y. Singh and P. Gegenwart, Phys. Rev. B 82, 064412 (2010).[24] X. Liu, T. Berlijn, W.-G. Yin, W. Ku, A. Tsvelik, Y.-J. Kim, H.

Gretarsson, Y. Singh, P. Gegenwart, and J. P. Hill, Phys. Rev. B83, 220403(R) (2011).

[25] S. Bhattacharjee et al., New J. Phys. 14, 073015 (2012).[26] Y. Singh, S. Manni, J. Reuther, T. Berlijn, R. Thomale, W. Ku, S.

Trebst, and P. Gegenwart, Phys. Rev. Lett. 108, 127203 (2012).[27] J. Chaloupka, G. Jackeli, and G. Khaliullin, Phys. Rev. Lett.

110, 097204 (2013).[28] M. Kargarian, A. Langari, and G. A. Fiete, Phys. Rev. B 86,

205124 (2012).[29] J. Reuther, R. Thomale, and S. Rachel, Phys. Rev. B 86, 155127

(2012).[30] F. Ye, S. Chi, H. Cao, B. C. Chakoumakos, J. A. Fernandez-Baca,

R. Custelcean, T. F. Qi, O. B. Korneta, and G. Cao, Phys. Rev.B 85, 180403(R) (2012).

[31] R. Schaffer, S. Bhattacharjee, and Y. B. Kim, Phys. Rev. B 86,224417 (2012)

[32] K. Foyevtsova, H. O. Jeschke, I. I. Mazin, D. I. Khomskii, andR. Valenti, Phys. Rev. B 88, 035107 (2013).

[33] T. Hyart, A. R. Wright, G. Khaliullin, and B. Rosenow,Phys. Rev. B 85, 140510(R) (2012).

[34] Y.-Z. You, I. Kimchi, and A. Vishwanath, Phys. Rev. B 86,085145 (2012).

[35] S. Okamoto, Phys. Rev. B 87, 064508 (2013).[36] L. Kimme, T. Hyart, and B. Rosenow, arXiv:1308.2496

[cond-mat.supr-con].[37] M. Sato, Phys. Rev. B 79, 214526 (2009).[38] M. Sato and S. Fujimoto, Phys. Rev. B 79, 094504 (2009).[39] M. Sato, Phys. Rev. B 81, 220504(R) (2010).[40] R. Comin et al., Phys. Rev. Lett. 109, 266406 (2012).[41] Y. K. Kim, O. Krupin, J. D. Denlinger, A. Bostwick, E.

Rotenberg, Q. Zhao, J. F. Mitchell, J. W. Allen, and B. J. Kim,Science 345, 187 (2014).

[42] W. Metzner, M. Salmhofer, C. Honerkamp, V. Meden, and KurtSchonhammer, Rev. Mod. Phys. 84, 299 (2012).

[43] P. A. Lee, N. Nagaosa, and X.-G. Wen, Rev. Mod. Phys. 78, 17(2006).

[44] C.-Y. Hou, C. Chamon, and C. Mudry, Phys. Rev. Lett. 98,186809 (2007).

[45] A. G. Grushin, E. V. Castro, A. Cortijo, F. de Juan, M. A. H.Vozmediano, and B. Valenzuela, Phys. Rev. B 87, 085136(2013).

[46] T. C. Lang, Z. Y. Meng, A. Muramatsu, S. Wessel, and F. F.Assaad, Phys. Rev. Lett. 111, 066401 (2013).

[47] S. Raghu, X.-L. Qi, C. Honerkamp, and S.-C. Zhang, Phys. Rev.Lett. 100, 156401 (2008).

[48] J. Wen, A. Ruegg, C.-C. Joseph Wang, and G. A. Fiete,Phys. Rev. B 82, 075125 (2010).

[49] N. A. Garcia-Martinez, A. G. Grushin, T. Neupert, B.Valenzuela, and E. V. Castro, Phys. Rev. B 88, 245123 (2013).

[50] M. Daghofer and M. Hohenadler, Phys. Rev. B 89, 035103(2014).

[51] T. Duric, N. Chancellor, and I. F. Herbut, Phys. Rev. B 89,165123 (2014).

[52] M. L. Kiesel, C. Platt, and R. Thomale, Phys. Rev. Lett. 110,126405 (2013).

[53] S. Sachdev and R. La Placa, Phys. Rev. Lett. 111, 027202 (2013).[54] J. D. Sau and S. Sachdev, Phys. Rev. B 89, 075129 (2014).[55] A. Allais, J. Bauer, and S. Sachdev, arXiv:1402.4807

[cond-mat.str-el].[56] A. Allais, J. Bauer, and S. Sachdev, Ind. J. Phys. (2014),

doi:10.1007/s12648-014-0488-4.[57] M. Sigrist and K. Ueda, Rev. Mod. Phys. 63, 239 (1991).[58] C. Platt, W. Hanke, and R. Thomale, Adv. Phys. 62, 453 (2013).[59] J. W. Negele and H. Orland, Quantum Many-Particle Systems

(Addison-Wesley, Reading, MA, 1988).[60] M. Salmhofer and C. Honerkamp, Prog. Theor. Phys. 105, 1

(2001).[61] A. Eberlein and W. Metzner, Phys. Rev. B 89, 035126 (2014).[62] J. Reuther and P. Wolfle, Phys. Rev. B 81, 144410 (2010);

J. Reuther and R. Thomale, ibid. 83, 024402 (2011); J. Reuther,P. Wolfle, R. Darradi, W. Brenig, M. Arlego, and J. Richter,ibid. 83, 064416 (2011); R. Suttner, C. Platt, J. Reuther,and R. Thomale, ibid. 89, 020408(R) (2014); J. Reuther andR. Thomale, ibid. 89, 024412 (2014).

[63] C. Honerkamp, Phys. Rev. Lett. 100, 146404 (2008).[64] H. Zhai, F. Wang, and D.-H. Lee, Phys. Rev. B 80, 064517

(2009).[65] W.-S. Wang, Y.-Y. Xiang, Q.-H. Wang, F. Wang, F. Yang, and

D.-H. Lee, Phys. Rev. B 85, 035414 (2012).[66] M. L. Kiesel, C. Platt, W. Hanke, D. A. Abanin, and R. Thomale,

Phys. Rev. B 86, 020507(R) (2012).[67] A. M. Black-Schaffer and S. Doniach, Phys. Rev. B 75, 134512

(2007).[68] F. Wang, H. Zhai, Y. Ran, A. Vishwanath, and D.-H. Lee,

Phys. Rev. Lett. 102, 047005 (2009).[69] R. Gersch et al., New J. Phys. 10, 045003 (2008).[70] M. Ossadnik and C. Honerkamp, arXiv:0911.5047 [cond-mat.

str-el].[71] M. M. Scherer, S. Uebelacker, and C. Honerkamp, Phys. Rev.

B 85, 235408 (2012).[72] T. C. Lang, Z. Y. Meng, M. M. Scherer, S. Uebelacker,

F. F. Assaad, A. Muramatsu, C. Honerkamp, and S. Wessel,Phys. Rev. Lett. 109, 126402 (2012).

[73] M. M. Scherer, S. Uebelacker, D. D. Scherer, and C. Honerkamp,Phys. Rev. B 86, 155415 (2012).

[74] C. Honerkamp, M. Salmhofer, N. Furukawa, and T. M. Rice,Phys. Rev. B 63, 035109 (2001).

[75] Z. Nussinov and J. van den Brink, arXiv:1303.5922 [cond-mat.str-el].

[76] P. Kopietz, L. Bartosch, and F. Schutz, Introduction to theFunctional Renormalization Group (Springer Verlag, Berlin,2010).

045135-21

http://dx.doi.org/10.1088/1367-2630/12/6/065010

http://dx.doi.org/10.1088/1367-2630/12/6/065010

http://dx.doi.org/10.1088/1367-2630/12/6/065010

http://dx.doi.org/10.1088/1367-2630/12/6/065010

http://dx.doi.org/10.1038/nature08917








http://dx.doi.org/10.1080/00018739500101566

http://dx.doi.org/10.1080/00018739500101566

http://dx.doi.org/10.1080/00018739500101566

http://dx.doi.org/10.1080/00018739500101566

















http://dx.doi.org/10.1016/j.aop.2005.10.005




















http://dx.doi.org/10.1088/1367-2630/14/7/073015

http://dx.doi.org/10.1088/1367-2630/14/7/073015

http://dx.doi.org/10.1088/1367-2630/14/7/073015

http://dx.doi.org/10.1088/1367-2630/14/7/073015









































http://arxiv.org/abs/arXiv:1308.2496










































































http://dx.doi.org/10.1007/s12648-014-0488-4





http://dx.doi.org/10.1080/00018732.2013.862020

http://dx.doi.org/10.1080/00018732.2013.862020

http://dx.doi.org/10.1080/00018732.2013.862020

http://dx.doi.org/10.1080/00018732.2013.862020

http://dx.doi.org/10.1143/PTP.105.1




















































http://dx.doi.org/10.1088/1367-2630/10/4/045003

http://dx.doi.org/10.1088/1367-2630/10/4/045003

http://dx.doi.org/10.1088/1367-2630/10/4/045003

http://dx.doi.org/10.1088/1367-2630/10/4/045003



















unconventional pairing and electronic dimerization ... · doi: 10.1103/physrevb.90.045135 pacs...

Documents