A new and universal scheme for passive decoy-state quantum key distribution (QKD) with the two-mode state source is proposed. As one of the two-mode

the trigger state can obtain four types of detection events after splitting and detecting at the transmitter. Based on those events

the other mode as the signal state is divided into four sets of pulses which can be used to estimate parameters and extract key. Besides

the performance analysis is carried out on the scheme with the heralded single-photon source (HSPS) and the heralded pair coherent state (HPCS). The impact of different detection efficiency is discussed. Moreover

the statistical fluctuation in the practical system is numerically studied. Our simulation results show that the performance of our scheme is superior to the existing three-intensity decoy-state QKD schemes with different sources in terms of bit error rate and secure transmission distance (up to 198.6 km). The performance of using HPCS is better than that of using HSPS

and the key generation rate increases with the rising of detection efficiency at the transmitter. Considering the statistical fluctuation

the efficiency of HPCS is also better than HSPS

and the maximum secure distance of our scheme can reach 164 km when the data length is 109. Furthermore

our scheme only needs to use a single intensity pulse

which reduces the difficulty of the system implementation while improving the system performance. It has certain reference value to the implementation of QKD system.