Shared hosting: slow, restrictive and overused
Good hosting can be cheap and flexible, don't comprise your website by sticking it on cheap shared hosting.
Shared hosting is when a hosting company sells a small slice of disk space on one big server to hundreds or thousands of users. Every request to every site on that server will be handled by the same web server and you have very little flexibility in what software you can install.
The nature of shared hosting is that it is shared. You don't want your site to be slow when another site on your server is getting hammered.
Many of the optimisations, tweaks and a lot of software can't be installed on shared hosting, limiting what can be done on your website.
Your data is on the same machine as a lot of other random websites and is also being constantly accessed by other users, increasing security risks.
This used to be the best solution available in the past when the only choice was affordable shared hosting or buying a very expensive dedicated server, but thankfully technology has moved on.
Unfortuantely, many companies still use shared hosting for legacy reasons or simply because it's all they're used to and easy for them to manage.
Virtualization: a better alternative
In recent years virutalization technology and hosting companies offering it have massively improved. Virtualization allows you to create small 'virtual' servers with their own operating systems on a large dedicated server.
In principle this might sound quite similar to shared hosting, but in practice it a virtual server (often referred to as a VPS) offers the exact same flexibility to install any software as having your own dedicated server. Want to install a faster web server or some new caching technology? No problem.
A VPS will normally come with a set of dedicated CPUs and memory, which means your virtual server won't get slow if another on the same physical server is using lots of resources.
Even better, many modern companies selling virtual servers allow you to pay-as-you-go, so you only pay for time used down to the nearest hour. Because they are 'virtual' they also be very easily scaled up or down. Starting to get more traffic, simply upgrade to the next tier with a click of a button.
More flexible than a dedicated server
The final big benefit of using virtual servers is that you can build very scalable, reliable and high-availability system at a fraction of the cost when using dedicated servers.
Buying a dedicated server normally requires a one or two year contract which can be incredibly expensive. You also have to predict how much traffic you will get so your server can handle it. If you end up getting no traffic you will be paying for the dedicated server anyway.
This is such an advantage that I would never recommend a client buys a dedicated server no matter how big they are or how much traffic they need to handle.
Pay per hour
Minimise the cost by paying for only what you need on an hourly basis.
Easily resize servers as you need to based on demand.
Build more advanced networks for high-availability.
Install any software, configure anything you need to.
Your data and files are on a different operating system.
Much faster serving of data when configured well.
For the price of a dedicated server it is possible to set up 5-6 smaller virtual servers with smart re-routing of traffic able to handle far more traffic. This means if one server goes down your site will continue to work fine, unlike if you had a single large dedicated server.