Honestly, yes it does affect you negatively.
I used to do mile repeats with my team at 5:04 pace and one day I messed up the workout by going 4:52, 4:56, 4:58, 4:54...
Afterwards, the coach was furious. I couldn't understand why because I thought I had done pretty well.
He then explained that workouts are supposed to be controlled runs that makes us better prepared for the real deal: races.
He said that by pushing myself than I had to in practice, I was harming my training more than bettering it because it would not help me in the long run.
Years now, I completely agree with what that man told me.