Imagine that you buy a really nice car that you just love. You drive it around for a few months, throw in a couple of aftermarket parts to make it even nicer, and all is well.
A few months later the manufacturer sends you a couple of recall notices. They seem relatively minor and/or you don't understand them, so you decide it's not worth the effort to take time off to get those things fixed.
A couple of years go by and you're still relatively happy with the car. You haven't changed the oil once because, well, your car is running just fine as far as you can tell.
Suddenly one day you start hearing a strange noise coming from the engine compartment. It's intermittent so it's not a priority. It still gets you to work every day. You ask your friend who knows a little bit about cars to take a look, and he throws a couple of bottles of additive in to the gas tank.
A few weeks later another friend is in the car with you and says "hey, what's that noise?" You're suddenly aware that your car creaks with every dip in the road. When you hit the gas the car seems to hesitate, or maybe the tires are slipping a little when you try to accelerate or brake a little too hard.
"Damn this city and their lack of road maintenance!", you might think. "Eh, this car just builds up sludge faster than other cars", says your not-a-mechanic-but-knows-enough-about-cars friend as he dumps another bottle of fuel system cleaner in to your gas. "There's nothing wrong with your car! You just need to drive exclusively on our roads!", says the local toll road authority (who, to their credit, actually does keep the road in pristine condition).
What would you consider to be the real problem here?
So, to get to my point:
It sounds to me like you don't have the knowledge or team to support to choice in platform that you've made. The problem isn't the host (AWS). The problem isn't the platform (WordPress). The problem is that you've installed "loads of plugins" on top of an aging installation and have not given it the love and maintenance that it needs.
Now, I don't want to be harsh here, but a 60 second scan of your site and server configuration gave me the impression of a project where prioritization decisions were misguided. AWS hosting comes in to play when you have a very strong technical team, i.e. an AWS SysAdmin who will properly configure your environment and firewall, a Server SysAdmin who will properly configure and secure the O/S for a public facing site, a DB SysAdmin who understands the needs of the applications it will be supporting and will configure it as such, etc. THEN, you need your WordPress Admin/Webmaster who will actually manage the WordPress site and/or its content.
If this sounds like a bit much, that's because it is. This is a WordPress site that doesn't require much more than a reputable webhost and the WordPress admin. If you're running on AWS and don't have a full-time sysadmin or someone who isproactivelymaintaining the server, you're doing it wrong. Likewise, if your site is crashing a lot and you continue to have to reach out to your webmaster to get it back up (reactive), you're doing it wrong.
I've written all this up not just in response to Stephen's post, but for some of the other non-technical founders I see here posting with similar issues. Don't be led in to thinking that signing up with the Cadillac of web-hosts is a good idea unless you also have a team of Cadillac experts available to you. A server is not a "sign up and forget about it" deal. I've taken on way too many projects where a previous technical "adviser" led them to some very expensive technical decisions that they had no team to support -usually because the previous technical person wanted to get their feet wet in a new technology at the expense of the client.
Okay, I'm done ranting. My advice to Stephen:
1.) Get away from AWS. The fact that your AWS server is running outdated software with known security issues and open service ports tells me that you don't really have a sysadmin on your team. (A WordPress expert or even a very strong Web Developer does not imply knowledge in System Administration.) Move to a reputable host. I've heard good things about Pagely, A Small Orange, and SiteGround, however have very little personal experience with them. My understanding is that under Pagely, security updates and maintenance items are applied globally and automatically by their team. (Edit: By the time I finished my reply, several others recommended WP Engine.)
2.) If not already done, get your guy to clone your site so you have a "test" version and a "live" version.
3.) Get your guy to update the WordPress installation to the latest versions. Remove any and all plugins (not just disable - REMOVE) that you don't need, and strongly consider which of your plugins are absolutely critical. This would be done on the Test site so that there is no risk to the Live site during this process. If all goes well, migrate it over to Live.
4.) SSL certificates are cheap. Get one and run the site over HTTPS. It adds a tiny bit of additional credibility (and may have a positive impact on SEO.)
5.) Add proactive maintenance to your strategy. A good host dedicated to WordPress may include managed services/maintenance as part of the plan. Let your web guy do what he's good at: design, content maintenance, etc, and let the infrastructure guys handle what they're good at.
I hope this helps you understand some of the issues from a higher level, and also helps push you in to seeking a very strong technical adviser to run any technical decisions by. In the end, you're really just running a basic, informational WordPress site so don't complicate things any more than necessary.