Bring milk and tea to a simmer in a small saucepan set over medium heat. Remove from the heat, cover, and steep for 30 minutes. Strain the infused milk into a liquid measuring cup, pressing a spoon against the tea leaves to extract as much liquid as possible. Keep 1 1/2 cups liquid for the cupcake batter and set aside the remaining (approximately 1/4 cup) for the buttercream. Cool to room temperature before using in the batter and frosting.
Preheat the oven to 350°F. Line two cupcake pans with liners and spray lightly with baking spray.
In a stand mixer fitted with the paddle attachment (or using a large mixing bowl with an electric hand mixer), cream together the butter, sugar, cardamom, cinnamon, ginger, salt, baking powder, and vanilla until light and fluffy, 5-7 minutes.
Add the eggs to the butter mixture one at a time, beating well after each addition. Swiftly alternate between adding the flour and 1 1/2 cups chai milk on medium-low speed, starting and ending with the flour. Scrape the sides and bottom of the bowl a few times.
4 large eggs, 2 large egg yolks, 2 3/4 cups all-purpose flour
Pour the batter into prepared pans and bake for 22-25 minutes. Remove from oven, cool for 10 minutes, then transfer to a wire rack to cool completely before frosting.
Frosting
In a small bowl, combine the cinnamon, ginger, cardamom, and salt. In a stand mixer fitted with the paddle attachment, combine the butter and powdered sugar. Add the spices and vanilla. Slowly add in the reserved 1/4 cup milk until the desired consistency is reached.