I think you are onto something here. I would try to find an angle that gets the traffic light out of the frame as it is distracting. The other thing I would suggest is maybe trying to do a long enough exposure that would give you light streaks from the cars. Not sure the effect of the lens baby is real predominate in this scene, maybe if you were shooting down a street where there were more lights or signs on the buildings the effect would be stronger?
Keep it up!
Looking at it again the blur effect on the left from the lens baby, for me, does help direct my eye back down the street so it is working a bit better than I thought at first.