This definition of startup by Eric Ries captures the qualities unique to a startup without limiting the scope to venture-backed technology companies:

A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty.