Shotaro Kaga designed the Museum's main building as his country villa, giving it the name Oyamazaki Villa. Originally built of wood in the Taisho era, it was expanded substantially in the subsequent early Showa period. Kaga visited in Europe in his youth and it was memories of gazing out at the River Thames from Windsor Castle in England which eventually led him to seek land in Oyamazaki, where the three rivers Kizugawa, Ujigawa, and Katsuragawa merge. Kaga started construction in 1912, with the first phase completed around 1917.

It is said that Kaga's inspiration for the original villa was a coal miner's house he saw in Britain, and this now remains the entrance hall of the present main building. He undertook renovation work in 1922 and the current main building is thought to have been completed in 1932. The upper part of the building's timber frames are in the style of an English Tudor cottage, while steel reinforced concrete and steel frames are used for the rooftop. From the second floor's terrace of the main building, today used as café, visitors are able to gain a magnificent view of the three rivers.

In order to regenerate the former Oyamazaki Villa as a museum, a new underground building was added, designed by the architect Tadao Ando. Formally known as the Chikukan, Ando named the annex the Underground Jewelry Box. He designed the building semi-buried underground, with greenery planted on top of the cylindrical gallery exhibition space to maintain harmony with the surrounding natural landscape.

The main building and annex are connected by a corridor. Visitors descend the stairs surrounded by high walls on both sides to arrive at the subterranean gallery. The use of glass on all four sides and the front of the stairway passage enables the visitor to view beautiful trees outside. In this gallery, masterpieces from Claude Monet's "Water Lilies" series are permanently exhibited.

In 2012, 100 years after the first work on Oyamazaki Villa began, a new gallery annex was completed, Dream Box, again designed by Tadao Ando.

Standing by the water-lily pond, the cube-shaped building makes a sharp contrast to the cylindrical Underground Jewelry Box. To avoid disturbing the landscape viewable from the main building's terrace overlooking the Seikaro tower in the villa grounds, Ando placed the building semi-concealed in the surrounding trees, with greenery also planted on the upper part of the structure. As a consequence, the straight lines of the concrete architecture blend well with the nature of Mt. Tennozan.

Where the Dream Box annex currently stands there was once the Oyamazaki Villa greenhouse, widely known for cultivating orchids, and the glassed corridor connecting the main building and the Dream Box was initially used as a passageway to the greenhouse. A new phase for Asahi Group Oyamazaki Villa Museum of Art thus starts from the spot where Shotaro Kaga formerly chased his orchid dreams.