the water in san jose sucks.
it depends on what you're looking for in terms of family housing. if you don't have kids living downtown wouldn't be too bad, but it wouldn't be my first choice as far as places to live. I lived a 20 minute walk from downtown for a few years until my first kid was school age...and then I bought a house in a good east bay school district.
also depends on where you're going to work in san jose. san jose is really fucking big.
downtown there's some tech companies, shitty schools and a lot of homeless people, but there's some good places to take small kids and some good places to eat.
up north along first street and tasman there's a lot of tech companies...altho somewhere in there you end up in santa clara and sunnyvale and mountainview.
ebay has another campus out by hamilton, which isn't a totally shitty area.
tbh all the cities out there have kinda merged together into one massive suburb. unless your goal is to bike/walk to work there's no reason to limit a housing search to only san jose just because you're working in san jose.
rents and housing prices are better in san jose than other parts of the bay.