Bought this great book of New York City historical maps today.
This is a map of "Mannados" in 1664. The northern edge of New Amsterdam (just in the process of changing its name to New York) was protected by a wall, and hence the street along the wall was called "Wall Street."
The next map is from 1766. The city has spread a bit north now, but still has a long ways to go. Population had only recently passed 10,000. The port was way behind Boston and Philadelphia and barely ahead of Newport, Rhode Island.
Way up in the upper right on the Hudson is a little hamlet called Greenwich, surrounded by lots of farmland. Directly east of this hamlet, which was roughly around where Hudson and Christopher Streets intersect today, is where I am typing these words right now.
The last map is from 1836, when the city has spread as far as around 14th street. In the summer of 1822, there was a yellow fever epidemic in New York, reflecting the poor sanitation in the city among other things. This was the last straw for the wealthier inhabitants who moved to a suburb that became known as Greenwich Village, named after the hamlet in the previous map. To maintain property values in the Village, an enterprising New York mayor named Philip Hone created a Washington Memorial Parade Ground (the land was available because it was a graveyard for the criminals and poor; Mayor Hone just paved over it; many remains are still buried beneath what is today's Washington Square).
In the map, directly beneath the number 15 is a little building on the west of Washington Square that housed the entirety of New York University in 1836.
Why did New York (and Greenwich Village) thrive so much more than other east coast cities? Cities are obvious cases of increasing returns, where the more you have already, the more new stuff you attract. (Many people have looked for development insights from this, without ever really resolving what they are.)
Another important event between map 2 and map 3 was the opening of the Erie Canal on October 26, 1825.
New York had major geographic luck -- the lowest point to cross the Appalachian Mountains was just west of Albany, so the Erie Canal could connect the vast interior linked to the Great Lakes to Albany and down to New York City. After the Erie Canal became defunct, New York had now had such a head start that it is still the premier East Coast city. (Historical neighborhood details from here.)
As Billie Holiday sang, "them's that's got shall have, them's that's not shall lose."