The process of using a computer to identify or recognize a target from Synthetic Aperture Radar (SAR) images with or without human interference is known as Automatic Target Recognition (ATR). The recognition of target is the process of discovering the location, pose or class of a target with a particular spatial signature by using a high spectral resolution remotely sensed images, which belongs to a particular kind of object (vehicle). In recent years ATR is particular interest in military applications such as infrared surveillance and target acquisition, unmanned aerial vehicles, and autonomous missile. The traditional architecture of automatic target recognition (ATR) for synthetic aperture radar (SAR) consists of three stages: detection, discrimination, classification and recognition. In the last few years Many deep convolutional neural networks have been proposed and used for SAR-ATR and have obtained state-of-the-art results in many computer vision tasks, plus shown improvement from time to time, but most of them classify targets from target chips which is found from SAR imagery, due to limited training images in SAR-ATR, CNN yielded over-fitting when directly applied to SAR-ATR. This paper proposes a novel deep convolutional learning architecture, called Multi-Stream CNN (MS-CNN), for ATR in SAR by leveraging SAR images from multiple views. By deploying a multi-input architecture that fuses information from multiple views of the same target in different aspects enables it to make full use of limited SAR image data to improve recognition performance. In addition, the Fourier feature fusion framework derived which allows unraveling the highly nonlinear relationship between images and classes. The model proposed in this paper performs all the three tasks in the SAR-ATR architecture. The proposed CNN will train using the Moving and Stationary Target Acquisition and Recognition (MSTAR) benchmark data set and to output scores of 10 classes.