In large stock pot or Dutch oven, heat oil over medium-high heat. Add carrots, onion and garlic. Sauté until onions are softened (about 5 minutes).
Stir in curry powder and cayenne pepper; stir to coat vegetables. Add in vegetable broth and potato.
Bring to a boil. Reduce heat; simmer until vegetables are tender (about 25 minutes).
Remove from heat. Purée soup with an immersion blender or in batches in a food processor or blender (return puréed soup back to pot).
Stir in milk; salt and pepper to taste. Heat through.