One of the main reasons why people use SharePoint is to find information - information stored in document libraries, lists, or other sources linked from SharePoint. In fact, because the majority of an organization's information is unstructured data, SharePoint provides a great way to allow employees, partners, and customers to access all the disparate assets across an enterprise.
However, to enable end users to store and retrieve information easily, a well designed taxonomy must be in place. Failure to do so would lead to poorly maintained sites, ineffective site navigation, site proliferation and duplication of information, and worst of all, extremely unsatisfied users. When was the last time you were happy with the mess you have on your local file system?
A well designed taxonomy requires planning. SharePoint provides the right tools to implement that taxonomy, but this is an exercise that IT must conduct in collaboration with their end-users. Wider is always better than deeper. Organizations must invest up front in setting standards, infrastructure, and training. In addition, the quality of the content should also be a focus for improvement - think about more meaningful document library structures, or more descriptive metadata or content types.
Also, be prepared to adjust if needed. Testing or fine tuning your taxonomy is not a fun activity, but if you need to do it for the benefit of your end users, it's a path you'll have to take. You'll just have to be prepared to have the tools to manage/change your navigation, move sites or site collections, move content, etc...