Preheat your oven to 350°F (175°C). Line baking sheets with parchment paper or silicone baking mats.
In a large bowl, whisk together pumpkin puree, sugar, vegetable oil, egg, and vanilla extract until smooth.
In a separate bowl, sift together flour, baking powder, baking soda, salt, and pumpkin pie spice.
Gradually add the dry ingredients to the wet ingredients, mixing just until combined. The dough will be soft but manageable.
Scoop tablespoon-sized portions of dough onto the prepared baking sheets, spacing them about 2 inches apart. Flatten each slightly with the back of a spoon or your hand.
Bake for 12-15 minutes or until the edges are set and a toothpick inserted in the center comes out clean. Allow cookies to cool on the baking sheet for 5 minutes, then transfer to a wire rack to cool completely.
In a small bowl, whisk together powdered sugar, vanilla extract, and 2 tablespoons of milk. Add more milk, a little at a time, until the glaze reaches a smooth, pourable consistency.
Once cookies are fully cooled, drizzle or spread the glaze over the tops. Garnish with cinnamon, if desired. Let the glaze set for about 30 minutes before serving.