I'd say the biggest letdown was the follow-up to Val getting the twins back and the start of Season 7. Seasons 4 thru 6 are really the peak of the show. I might even throw Season 3 in there, too.
Season 7 was pretty drab overall as I recall, but I think they got it back on track, especially for specific storylines, such as Olivia's drug addiction and Laura's exit.
"Noises Everywhere" is a beautiful two-parter, and I can't watch it without thinking of the cast cozying up at David Jacobs' house for the weekend and the writers watching them interacting in character concerning Laura's death.
Season 9 did drag in spots, though, but Seasons 10 and 11 were very strong, I'd say.
Seasons 12 thru 14 just seem to have changed the atmosphere so much to me. I've heard others compare those final three seasons to daytime soaps, and they're not wrong in drawing that comparison. The writing slacks the most for Season 13, and even though Season 14 "fixed" a lot, it still seems quite fractured with Val gone and the absence of so many major characters for several episodes.