Make the Lemon Garlic Dipping Sauce: Mix the mayo, lemon juice, garlic powder and cayenne pepper together in a small dish. Season to taste. If making in advance, cover the bowl and store in the refrigerator for up to 2 days.
Wash the peppers and pat them dry with a towel.
Poke each pepper once with a fork. This step is optional, but it cuts back on the danger of popping peppers in the skillet.
Heat the oil in a heavy bottomed skillet over medium-high heat. When the oil is shimmering, it’s hot enough. In batches, add the peppers to the hot skillet, being careful not to overcrowd the skillet. Cook until brown and blistered on one side, then turn until blistered and browned on all sides.
Set the peppers on a plate. Sprinkle with flakey sea salt and a squeeze of lemon. Serve immediately with the dipping sauce.
