Good question! Most of these pictures are not using special filters, just a fancy camera! You can actually take pictures of the night sky with any camera you have that can take long-ish exposure images - from a glance over a few pages, it seems that 30 seconds is about as long as you can go before the internal electronics of the camera start adding noise to your image. From the ground, 30 seconds is actually slightly too long for really crisp images of the sky, so 10-15 second exposures are more frequently used. Then you can take a whole pile of them, one immediately after the other, and you can stack the images on top of each other. The stars, which exist in all of the images, should pop up more brightly in the stack!
The really impressive images, with hundreds of thousands of stars, often take the camera and attach it to a telescope mount. The telescope mount can move as the earth spins to correct for the motion, so you can take more and more images of the same patch of sky, and make a larger stack of images, which will allow you to see fainter stars!
The other thing you can do with the camera and no filter is take star trail pictures. Those are where you intentionally let the earth spin and smear the stars out into long streaks - like the time-lapse photos of the oceans where the waves have blurred into a smooth foam. The one above is taken from the International Space Station, so there’s light streaks from the Earth below as well. (You can take these from the ground too, I just really like this picture.)