I've come to the conclusion that even though it seemed like it would turn out to be a decent standalone story at the beginning, you really need to read Fate/stay night before reading or watching Fate/Zero (as was intended). A lot of the characters get their development in Fate/stay night and everything in Fate/Zero is meant to supplement it. So now that it's over I get to wall of text about this stuff.
Kiritsugu's character is introduced in Fate/stay night from the perspective of Shirou. We know he saved Shirou and is his adpotive father and that he's mellowed out. One of his key scenes in Fate/stay night is the very last scene in Fate/Zero. But the important thing relating to Fate/Zero is that we get hints that he's done things that he regrets and that ultimately, he realizes he was wrong. I saw some people complaining about this when he was arguing with Saber, but I think that by the end of Fate/Zero, it was pretty clear. Even so, if you've played Fate/stay night, you won't misread this point, because it's one of the main questions: you want to save everyone, but you can't, so who will you save?
It's important to note that something that Fate/stay night people will know that Fate/Zero didn't address is that Kiritsugu doesn't destroy the Grail because he it didn't do what he wanted to do. It's because he realized the Grail was tainted. I could get into how this happened, but the result is that any wish granted by the Grail will now inadvertently result in tons of people dying.
Unlike Kiritsugu, Kirei shows up in Fate/stay night and plays a huge role. He's the main big bad and gets a ton of development. So Kirei's story in Fate/Zero is more of an origin story. His motivations are completely inscrutable unless you've gone through Fate/stay night.
Saber in Fate/Zero is interesting because she only alludes to the Fourth war once in Fate/stay night, which is to mention that Kiritsugu only spoke to her three times. Her story in Fate/Zero stems from her seemingly odd choice of wish in Fate/stay night. If you paid attention, you'll notice that her wish at the beginning of Fate/Zero isn't the same. That said, I found Saber to be the least interesting in her own route in F/sn and thought she was much more interesting in UBW.
There's no assumption that viewers/readers are familiar with Arthurian legend, because it assumes that you've read Fate/stay night, which covers whatever material you need to get this part. After all, a huge part of Fate is Saber explaining and trying to justify her wish to Shirou.
So you're right that this show is really bad for casual fans, but that's the nature of Type-Moon works and unfortunately, this is a work that builds upon that kind of stuff.