Filling: In a medium bowl, mix together peanut butter, powdered sugar, salt, vanilla extract, and whole milk until smooth, then set aside.
Cookies: Preheat your oven to 350°F (175°C) and prepare a baking sheet with parchment paper.
In another bowl, whisk together flour, cocoa powders, pudding mix, powdered milk, salt, and baking soda.
In a large bowl, cream together granulated sugar and butter for 1 minute.
Add the egg, then the egg yolk, milk, and vanilla, mixing until just combined.
Gradually incorporate the dry ingredients, using your hands if necessary to avoid overmixing.
Fold in Reese's Pieces and Peanut Butter Chips.
Scoop dough into 9 portions, flatten, and place a ball of filling in the center.
Seal the dough around the filling and roll into a ball.
Place on the baking sheet, spacing them apart. Optionally, top with more Reese's Pieces and chips.
Bake for 10-12 minutes until edges are set.
Cool on the pan for 15 minutes before transferring to a wire rack to cool completely.