As a D50 user (the D40 replaced the D50), I feel that the D40 is a step down. Here's the reasons why.
1.) There's no top LCD. I use that top LCD for everything. The D40 put all the info on the main LCD. They got rid of the top LCD to make it smaller.
2.) There's only 3 focus points. The D50 has 5. I was playing w/ my friend's D40 last month, and for the life of me, could not figure out how to select the focus points, which makes me believe that you can't at all. The D50 lets you choose which focus point you want to use.
3.) The biggest problem is that there's no internal focus motor, which means that you can't use 95% of the lenses that have been made for nikon. As time goes on, that'll get a little bit better. TECHNICALLY, you can still use them, but there's no autofocus when using these lenses. Again, this handicap was put in place just to make the camera a little bit smaller.
Typically when you replace a model with a newer one, it's supposed to be better. But because of those 3 reasons listed above, I feel the D40 is even worse than the D50.
That's some of the reasons the D40 is hated on so much