I haven’t read Nemesis, so here’s my googled summary of the situation you’re referring to. In Nemesis, one of the characters discovers a previously unknown red dwarf star, only two light years from the Earth. This star is dubbed Nemesis, and, in the book, is on a path to mess with the orbit of the Earth such that the population of Earth needs to move.
Here’s the thing with both dust clouds and red dwarfs - they’re not totally invisible. A red dwarf star is fairly bright in the infrared, as dim as it might be in the optical - it only produces a little light that we can see, but far more as heat. If the star weren’t obscured with dust, it would be easily spotted by one of our all-sky surveys, as a weird bright source moving across the sky. Barnard’s Star, which is nearly 6 light years away, visibly moves through the sky - the gif below shows its motion over a 20 year period. There are a number of surveys hunting for moving objects, and though their primary target is usually asteroids, they’d still pick up on something like this.
This motion means that the star can’t be coincidentally behind a convenient dust cloud that happens to be along the line of sight between us and that star. It would quickly move out from behind the blocking cloud. (As a further point, our solar system isn’t in an area of the galaxy that’s particularly full of dust clouds, so it’s unlikely it would have anything to hide behind in the first place.
In any case, this means that if we’re going to have a star buried in dust and therefore invisible, it’s going to have to be carrying the dust with it. The easiest way for that to happen is if it never got rid of the dust cloud that it formed in - however, red dwarf stars are usually extremely old, and would have had time to burn away the dust that surrounds them, so this is also unlikely.
Even if it did somehow have a cloud of dust surrounding it, dust glows, if you’re looking at the right wavelength. The image below is the glow of carbon monoxide in our galaxy as taken by the Planck space telescope. Carbon monoxide only exists in very dense clouds of gas, the sort of dense cloud you need if you want to form new stars - any less dense and the molecule will be destroyed. We should be able to spot anything that’s very dense through one of these maps, if we hadn’t noticed it through its motion.
So I would say that the odds are pretty low that there’s a surprise red dwarf anywhere near our solar system. The odds are much higher that we’ll run into danger from within our own solar system - asteroids which cross the orbit of the earth are pretty common.