Getting started with New York: where do I begin?

Budget is limited, so I'd rather learn from your mistakes than make them myself.