Archaeologists believe the site near Kilmarnock is 6,000-years-old and was settled as man moved away from nomadic existence towards farming the land.
The discovery has been described as one of the most important of its kind in recent years.
Excavations have unearthed a number of post-holes which formed part of a rectangular building and fragments of Neolithic carinated bowl, used for cooking and storage.
It is thought that they date to around 4000 BC.
The rectilinear hall, which measured 14m in length and 8m in width, belonged to a type of house built by the first farming communities in Scotland.
Kenneth Green, excavation director at GUARD Archaeology of Glasgow, who carried out the archaeological work for Scottish Water, said: “This is one of the most important discoveries of this type in south west Scotland in recent years.
“Heavily truncated by millennia of ploughing, only the deepest parts of some of the post-holes survived, arranged in a rectangular plan and containing sherds of early Neolithic pottery, hazelnut shell and charcoal.
“The width and depth of these post-holes indicated that they once held very large upright timber posts, suggesting that this building was once a large house, probably home to an extended family or group of families.
“Up until this time, during the earlier Mesolithic period (c. 8000-4000 BC), Scotland was inhabited by small groups of hunter gatherers, who led a nomadic lifestyle, living off the land.
“The individuals who built this Neolithic house were some of the earliest communities in Ayrshire to adopt a sedentary lifestyle, clearing areas of forest to establish farms, growing crops such as wheat and barley and raising livestock such as cattle, sheep, goats and pigs.”
The archaeologist have been working with Scottish Water to identify sites of potential interest along the route of a new water route.
Evidence of an old water course was found very close to the Neolithic house.
GUARD Archaeology’s operations manager Warren Bailie said the site for the house, built on a small hill. was likely chosen given its close proximity to the water.
Andrew Grant, environmental adviser at Scottish Water, said he was “delighted” that something of such importance had been uncovered.
Mr Grant said: “As part of the project planning, Scottish Water identified the possibility of archaeology and so factored in time for the area to be excavated.
“However, the discoveries are even more significant than we had expected and we are delighted that, with the archaeologists’ help and expertise, we have been able to uncover something of such importance.”
The discoveries have been removed for recording and analysis. They will be claimed by the Crown and deposited in keeping with Scottish legal requirements as set out in the Scottish Government’s Treasure Trove Code of Practice.