In the mid-12th century the settlement at Castle Acre was surrounded, where not protected by the castle, with a massive earth bank and a deep ditch, enclosing a roughly rectangular area of flat ground covering 3.8 hectares. These defences are particularly well preserved on the west side. Property boundaries and the street pattern were probably re-ordered at the same time, aligned on what is now Bailey Street, which ran between the main gates.
In about 1200 new gatehouses were built, one of which is known only from an 18th-century observation. The other, the Bailey Gate facing Stocks Green, largely survives. It consists of a pointed archway flanked by solid semi-circular turrets, behind which the gate-passage could be closed with gates and a portcullis. When built it was flanked by the earthwork rampart (roughly on the line of the existing houses) and approached by a timber bridge across the defensive ditch (now filled in).