Hastings was at first ill-equipped to cope with the tidal wave of wealthy visitors, who initially could only stay in the few hotels or lodge with the residents of the Old Town. Speculative building of high class boarding houses and homes soon began, however. The visitors wanted to reside near the sea, and, as the big local landowners refused to sell most of the land around the Old Town for building, expansion could only take place westwards, under the Castle Cliff and into the then rural Priory Valley (now the town centre). But first the Castle Cliff had to be cut back, destroying some of the Castle in the process, in an enormously expensive operation only made viable by the anticipated rents from the rich visitors. By the late 1820s the fashionable Pelham Place, Pelham Crescent, Breeds Place and Wellington Square had been built. In 1828 James Burton began constructing the new town of
St Leonards, designed exclusively for the wealthy, and by the mid-1830s the two towns were growing rapidly towards each other along the seafront. Unfortunately, by this time Hastings had also dropped out of favour with the nation's fashion-leaders, although it remained popular with the middle classes for several decades.

The rapid westward expansion of Hastings in the 20 years after 1815 pointed clearly to the Priory Valley becoming an alternative central focus for the borough, to the detriment of the Old Town. Another omen of the future came in 1835 when the government reformed the administration of most towns, including Hastings, taking power away from the old closed elites and giving it to the new middle classes of traders, merchants, builders, etc. Henceforth, there were to be councillors, elected by the better-off residents every three years in a rather more democratic system - but not one that favoured the fishermen, as the Council was weighted in favour of the middle class western part of the borough. This helped to reinforce the antagonism that was developing between the 'East End', as the Old Town became known, and the West End, This was to have very serious consequences in the early 1880s.