Abstract. ATM card fraud is causing millions of naira in losses for the card payment business. The most accepted payment mode in today’s world is ATM card for online and regular purchasing; hence frauds related with it are also increasing. To find the fraudulent transaction, this study proposes a hidden Markov Model (HMM) based on the Poisson distribution (HMM[Pois]), the generalized Poisson distribution (HMM[GenPois]), and the Gaussian distribution (HMM[Gauss]) with the forward-backward algorithm which detects the fraud by using customers spending behavior. The proposed estimation procedure based upon the three distributions for the HMM model is used to construct a sequence of operations in ATM card transaction processing, and detect fraud by studying the normal spending behavior of a cardholder, followed by checking an incoming transaction against spending behavior of the cardholder. If the transaction satisfies a predefined threshold value, then the transaction is decided to be legitimate else, the transaction is declared as fraudulent. The evaluation statistics used shows that the HMM[Gauss] is the most appropriate model in detecting ATM card fraudulent transactions.

Keywords: Hidden Markov Model, Gaussian distribution, generalized Poisson, optimal detection, forward-backward algorithm, estimation, simulation.