Hey Quoc -from HTML5 onwards, this is a breeze just using off-the-shelf components. Specifically, check out http://www.videojs.com/ for an OSS embeddable player, convert videos to H264 .mp4 & webm to cover all devices ( http://en.wikipedia.org/wiki/HTML5_video#Browser_support ), and use either your own servers, or S3 + Cloudfront for hosting.
If you need any help implementing this, shoot me a PM.