Udayagiri is about 4 km from the modern town of Vidisha and about 13 km from the Buddhist site of Sanchi. Udayagiri is best known for a series of rock-cut sanctuaries and images excavated into hillside in the early years of the fifth century CE. The most famous sculpture is the monumental figure of Visnu in his incarnation as the boar-headed Varaha. The site has important inscriptions of the Gupta dynasty belonging to the reigns of Chandragupta I (c. 375-415) and Kumaragupta I (c. 415-55). In addition to these remains, Udayagiri has a series of rock-shelters and petroglyphs, ruined buildings, inscriptions, water systems, fortifications and habitation mounds, all of which have been only partially investigated.
Technically speaking, these are not actually caves. These are massive carved structures made during the Gupta period (4th-5th cent.). Most of these sculptures are dedicated to the Hindu God Vishnu and his different incarnations. This place shows the skill of the artists who worked during the Gupta rule. The Udayagiri Caves are an early Hindu ritual site and is currently a tourist site under the protection of the Archaeological Survey of India.