At that time the largest and most influential barony in southern Scotland was The Crawford Baronry. The Barony was established before 1100, when records of the period show Thorlongus of the Merse as Overlord of Crawford. From this line descended the surname of Crawford as the original occupants of the barony. Crawford Castle was in existence by 1175, and was probably built as an earthwork castle some time before this by the Crawford family.
The Lindsay family inherited half of the Barony of Crawford, known as Crawford Parish, via a marriage in 1215 to the elder daughter of Sir John Crawford, who died in 1246 without male issue. The Crawford family retained the other half, known as Crawfordjohn Parish, as the Barony had been divided among the Crawford family four generations earlier. Crawford Castle is located in Crawford Parish. From an early date, the Clan Carmichael of Meadowflat acted as hereditary constables of the castle, retaining this post under successive owners.
The castle was occupied by the forces of Edward I of England during the Wars of Scottish Independence. Between Christmas 1296 and the spring of 1297, William Wallace rode with John Graham and forty men to assault the stronghold, then known as Crawford Lindsay. Wallace stormed the castle and took it from the English garrison. Wallace had a personal interest in regaining the castle, as his mother, Margaret Crawford, was a daughter of the Clan Crawford Chief Hugh Crawford, who was then Sheriff of Ayrshire.
In 1398, Robert II granted the title of Earl of Crawford to David Lindsay, who had won great praise on St George’s Day, 23rd April 1390 for bravery in a duel with the Englishman Baron Welles on London Bridge after Welles, at a banquet in Edinburgh, and presumably after too much alcohol issued, as Champion of England, the challenge: "Let words have no place; if ye know not the Chivalry and Valiant deeds of Englishmen; appoint me a day and a place where ye list, and ye shall have experience."
At the accession of James IV in 1488 the barony of Crawford was transferred to Archibald Douglas, 5th Earl of Angus for supporting his father, James III, against the young prince's rebellion. The Earls of Angus held the castle until 1578, when their estates were forfeited by the young James V. James used Crawford as a hunting lodge until his own death in 1542. His mistress, Elizabeth Carmichael, was the daughter of the hereditary constable.
After 1542 the barony was returned to the Earls of Angus, the keepership of the Carmichaels of Meadowflat coming to an end in 1595. In 1633 the 11th earl was created Marquess of Douglas, and the castle was probably rebuilt after this date. The castle then passed to the Duke of Hamilton, before being sold to Sir George Colebrooke in the 18th century. After a period of use as a farmhouse, the building was abandoned at the end of the 18th century, and much of the stone reused to build the present Crawford Castle Farm. Four stone tablets bearing coats of arms, one with the date 1648, are built into the west and south walls of the Castle Crawford House.
There is also the Craufurdland castle that is fully restored and privately owned. There are many more that changed names after the crawford widows remarried and their estates took up the husbands name, such as Cambell, Lindsey, Duncan...