Visual relationship detection is an intermediate image understanding task that detects two objects and classifies a predicate that explains the relationship between two objects in an image. Relations among entities play a central role in image understanding. Due to the complexity of modeling (subject, predicate, object) relation triplets, it is crucial to develop a model that cannot only recognize seen relations but also generalize to unseen cases. Inspired by a previously proposed visual translation embedding model, the context-augmented translation embedding model can capture both common and rare relations. The previous Visual translation embedding model maps entities and predicates into a low-dimensional embedding vector space and learns the embeddings guided by the constraint predicate, union(subject, object). In addition to the framework is combined with a language model that learns which relationships are more suitable between pairs of object classes. The model solves the scene graph inference problem using the standard Bi-GRU model and learns to iteratively improve its predictions. Our joint model can take advantage of contextual cues to make better predictions on objects and their relationships from an image. This work explicitly models the objects and their relationships using scene graphs, a visually grounded graphical structure of an image. Where in the graph, the nodes are represented as the objects, and the directed vertex is represented as relationships. The experiments show that our model significantly outperforms previous methods on generating scene graphs using the Visual Genome dataset.