Recognizing the patient's posture and physical actions is the key focus on their rehabilitation to restore or enhance the functional and motor abilities of those with physical disabilities. In this paper; we focus on the gross motor skills in adults who are in rehabilitation from an injury and will present a method for recognizing their posture and activities using a sequence of RGB-D images. The shape features are extracted using the depth information in the frequency domain via spherical harmonics representation.