I think you really need to understand how to use the light that is around at the time you are shooting. While the times you describe may be best for certain types of shot, bear in mind that the real harsh light of midday can, on occasion, produce some really interesting shots. Its all about how you use it I guess. Having said that, I'm all for breaking rules