In a large bowl, whisk flour, cornstarch, baking powder, and salt.
In the large bowl of a stand mixer fitted with the paddle attachment (or in another large bowl, using an electric mixer), beat granulated sugar and butter on medium speed until pale and fluffy, 2 to 4 minutes. Add milk, vanilla, and almond extract and beat until combined. With mixer on low speed, gradually add dry ingredients and beat, using your hands if needed to push together, until just combined; do not overmix.
Divide dough in half and form into 2 discs. Wrap each in plastic wrap and chill until firm, 2 to 3 hours. (Alternatively, divide dough in half and roll each half between 2 pieces of parchment to 1/4" thick. Chill until firm.)
Preheat oven to 350°. Line 2 large baking sheets with parchment. On a lightly floured surface, roll one disc to 1/4" thick. (Alternatively, peel off top sheet of parchment from dough.) Cut out desired shapes with a cookie cutter and arrange on prepared sheets.
Bake cookies until slightly puffed and set, 8 to 10 minutes, depending on size of cookie. Let cool on baking sheets 5 minutes, then transfer to a wire rack and cool completely.