Banks are another option you can look into. They're not open as many days and hours as grocery stores or Walmarts, but you're going to find their branches a lot more frequently, as there will usually be at least one even in a small town. They're not likely to have anything else you need, but you can get stamps.
Gas stations are another potential option. On the plus side, their hours are typically much better than banks, and there are always gas stations around. On the other hand, while many of them sell stamps, a lot of them actually won't. It really is just up to each individual gas station if they're going to do it or not.
Grab-and-go stores with a pharmacy often carry stamps at their primary cash registers up front near the door. Not all of them do it, but major chains like CVS, RiteAid, and Walgreens should be there for you.
It's still a new thing, but some ATMs sell stamps if they're newer machines. At the time of writing, this is mostly a West Coast thing, but it's spreading as time passes on. More info here https://wheretobuystamp.org/
