Preheat the oven to 350F (180C).
In a medium non-reactive pan, melt the butter over medium heat. Stirring the butter frequently, cook it gently until it begins to smell fragrant and nutty, and the butter begins to brown. Once the butter is brown, remove it from the heat and allow it to cool completely while assembling the rest of your ingredients.
Combine the all-purpose flour, baking powder, baking soda, and salt, and set aside.
Cream the cooled brown butter and the brown sugar together in a stand mixer fitted with a paddle attachment until it’s light and fluffy. Add the eggs one at a time, and mix just to combine, scraping down the sides as necessary. Add the zest, orange juice, olive oil, and vinegar, and mix just to combine. Add the dry ingredients in two additions, mixing just to combine.
Scrape the batter into the bottom of the prepared pan, leveling it so it covers the caramel portion completely in an even layer.
Bake for 45-50 minutes, rotating the pan once halfway through baking. When the cake is done, it should be an even deep golden brown, and the cake should spring back lightly when pressed in the center.
Allow the cake to cool for ten minutes (if you let it cool much longer, the caramel portion will stick to the bottom of the pan when unmolding). Using a knife, gently separate the edges of the cake from the pan.
Using potholders or a towel, flip your cake over onto your serving plate and remove the pan. If any of the prune slices stick to the bottom, you can simply press them back into place on the surface of the cake. Allow to cool for about 10-15 minutes further before slicing.
Serve plain, slightly warmed, as a treat at breakfast, or dress it up with whipped cream or vanilla ice cream as a dessert.