Stone and brick are not good insulation. At all. Feel a stone in the shade: it's cool. Feel a stone in the sun: it's warm. That's because stone, and brick to a slightly lesser degree, rapidly transfer heat. The opposite of insulation. Adobe can keep a shaded dwelling cool but build an adobe house in the sun and you'll feel the heat. To about the same degree as stone. And adobe is only usable in areas that don't get a lot of rain., unless you don't mind the exterior walls of your house being turned into mud.

Don't build with adobe along the east coast of around the Gulf. Ever.