I wouldn’t worry so much about insulation, as they can build their own nests, but it does sound like the sleeping room is a bit on the small size. Might be suitable for a juvenile, barely. I would remove the walls as you said. I read somewhere the nest boxes should be around 30cm by 40cm, so if you turn it into one large chamber with an entrance tunnel that should be close enough. Most of mine are closer to 30cm by 30, though, and they still get used.
Is the hog house off the ground at all? Flooding can be a major problem with all the rain we’ve had.
