Yes - find a pot with a drainage hole in the bottom, almost all plants require drainage, very few will grow well in stagnant conditions.

There are quite a few DIY sites that will tell or show you how to drill drainage holes into the bottom of the glazed or unglazed clay pot and this can be done with the plant in it if you care to go that route.

To drill a drainage hole into a clay pot or ceramic, you need a diamond tip drill bit and of course a good power drill. Be patient, it can take a long while to drill through the pot. Keep lubricated with water to avoid friction heat damage to your diamond tip drill bit. They aren't cheap.

Once you get the hang of it, you can get creative and turn all kinds of things into interesting flower pots: old coffee mugs (my African violets), tea pots, ceramic cookie jars, odd bowls and dishes. Just drill a drainage hole in the bottom....BOOM! Instant flower pot. It's fun! You can get them at Home Depot or Lowes. It's a handy addition to your toolkit.