Actually, you do have to pay a franchise tax if you incorporated in DE, whether that's a place of business or not. But for LLCs (at least small ones), the DE franchise tax is $200 (or $250, I don't remember).
But I agree that it's probably the best location to set up your company if you plan to raise money or even outsource your accounting and taxes to a local professional (unless you incorporate in your home state, in which case you'll have no problem finding someone local to take care of those admin tasks).
Side question: is it correct to talk about "incorporating an LLC" or does the verb only apply to a C-Corp?