I nearly gave up on Schitt's Creek after 5 episodes or so (as I disliked all the characters) but stuck with it and it ended up being one of my favourites comedies for last few years.
Not sure at what point you should give up but I'd imagine most people have the same initial negative impression.