I'm just starting on this, so I'll jot down notes as I go along.
There are three options for starting a company in Hong Kong: a sole proprietor, a limited partnership, or a limited liability company (LLC).
There's a summary of the differences between them on this website. Though as you read their final recommendation, keep in mind that their business is to sell you a service to setup a limited company!
In my case I'll be a one-man band, so cross 'partnership' off the list. Then sole-proprietor, or LLC? My business will be small and simple, really just a way to accept payment for occasional projects I run. I'm looking for something very simple to setup and manage, am not over worried about liability, and don't expect to raise capital.
I'll choose to be a 'sole proprietor'.
My next step is to register the company with the Inland Revenue Department. I'll need to fill in a simple form at the Business Registration Office (4/F Revenue Tower, 5 Gloucester Road, Wan Chai), show them my ID card, pay a fee (currently HK$450 for one year), then 30 minutes later I can collect my Business Registration Certificate. More details at the IRD website.