4 cups powdered sugar, or more for a thicker frosting
1/3 cup whole milk
1/4 cup unsweetened cocoa powder
To make the brownies: Preheat the oven to 350°F. Spray a 9x13-inch pan well with cooking spray.
In a small saucepan, combine the butter and cocoa. Set over low heat until the butter melts, then whisk together with the cocoa. Set aside to cool, approximately 5 minutes.
In a stand mixer fitted with the paddle attachment, combine the granulated sugar, flour, and salt. Add the cocoa/butter mixture and beat on medium speed until well blended, about 2 minutes. Slowly add the eggs and vanilla and mix until well incorporated.
Spread the mixture evenly in the prepared pan. Bake until a tester inserted into the center comes out with a few moist crumbs, 20 to 22 minutes.
Let the brownies cool completely, about 30 minutes.
To make the topping: Soften the peanut butter by placing it in a microwave-safe bowl and microwave for about 15 seconds. Stir the peanut butter well to distribute the heat evenly. Spread it on top of the brownies. Freeze the brownies for at least 30 minutes.
To make the frosting: In a medium saucepan, melt the butter and marshmallows over medium heat. Add the powdered sugar, milk, and cocoa and stir until smooth. (For a thicker frosting, add a little more powdered sugar.)
Spread the frosting over the peanut butter and freeze for 30 minutes.