Exoplanets (planets orbiting a star other than our own) have been much more present in the public eye since the launch and tremendous success of the Kepler planet-hunting satellite, which was really very recent; Kepler was launched in 2009, and had a few solid years of observations of a single patch of the Milky Way. But that’s not to say that scientists weren’t looking for them before then; the first tentative discoveries that were later confirmed to be real exoplanets happened in the late 1980s.
These early discoveries were slow, painstaking measurements, and had to focus on individual stars for long stretches of time, to be sure that a planet was the best explanation for whatever signal you saw in the light. What hashappened recently is that we’ve found a way to be much more efficient in our searching, so the number of known planets outside our solar system has skyrocketed.
To find an exoplanet, there a couple of different methods you can try, and in either case, the actual measurement that you need to take is reasonably straightforward in principle. In practice, as with many things where the Universe is involved, this will pretty rapidly get more complicated.
One of these methods is to look at the spectrum of the star you suspect has a planet. There are characteristic absorption & emission features in the spectrum of light that we observe; if we notice motion of those absorption and emission features, then we can plot those shifts over time. If there is a planet, you expect the star to wobble a little bit, because the gravity of the planet will tug on the star a little bit, very slightly pulling it off center. It’s this wobbling motion that you detect by watching the spectrum of the star itself. However, you have to watch the star for a long enough time to watch the star wobble a few times, which means you have to be taking data on your star pretty regularly (once every few days, in all likelihood) in order to be sure that you’re catching any kind of orbiting planet that’s possible. This method tends to be most effective at catching really massive planets, very close to their parent star, since they’re the ones that pull the most on your star, and the bigger the pull, the easier it is to measure that wobble.
Another common measurement is to just stare at the star with a very sensitive camera, and watch for changes in brightness. If you have a change in brightness - maybe the star seems to dim by 10 percent - and if that change repeats itself regularly, then what you’re looking at is likely to be an object passing in front of the star, which produces less light than the star itself. Doing this from the ground means that your camera has to be astonishingly well calibrated to be sure that any changes to the brightness of the star aren’t due to changes in the atmosphere, and even then, you’re most likely to find planets which are large and therefore block a lot of light. (Note: large in this case doesn’t necessarily mean they also have to be particularly massive, though at least with planets they often go together.) Again, this means taking regular measurements of individual, probably nearby stars, over a long period of time, so that you can catch the repeated flickering of the star’s light as it’s blocked.
If you want to avoid the atmosphere, then you have to put your telescope in space, which suddenly makes your planet hunting endeavor much more expensive. But this is where the Kepler satellite comes back into the picture. Kepler, orbiting the sun, avoids the Earth entirely, and points at a patch of sky away from the Sun. Kepler has one of the most fantastically precise cameras attached to it, which is able to detect changes to the brightness of a star’s light to much greater detail than ever possible on the Earth. Kepler can also monitor a huge number of stars at once; instead of focusing on an individual star, Kepler can monitor 100,000 stars at once. Ground based telescopes are not really designed to do this - they’re meant as multi-purpose sky observers, whereas Kepler was designed exclusively to find these fluctuations; Kepler is much more a single-purpose device.
So to go back to your question, of why this has only picked up recently – this has been an ongoing project of the scientific community for many decades. It’s just that these measurements are remarkably difficult to make from the ground, and we only recently got a planet-hunting satellite to go observe a fraction of our galaxy for a while. Once we got a device that’s custom-built to hunt for planets, then we were able to - so far – identify over a thousand planets orbiting distant stars.