Where are all the worms?

Hi, I garden in central Mass., zone 5, at my part-time home and chair the landscape committee at a condo complex where I live the other part of the time.

I've been growing veggies in large self-watering pots for a couple of years and am having very mixed results this year. From seed: peas, OK (until last week's 90-degree days); carrots, OK; spinach, holding at about 1"; chard, died off at about 1"; beans, holding at about 6". From seedlings: romaine, OK; red leaf lettuce, OK but overshadowed by romaine in same pot; parsley, great; basil, OK; sage, sad & struggling. The tomatoes I bought in 4" pots from a reputable nursery are not doing well at all. I'm wondering if I can grow mint in one of my large pots as a permanent thing, as I do chives (I put the pot in the crawlspace over the winter and bring it out when there are new green shoots).

In the ground, three Knockout roses in their 3rd year are looking chewed up and sad, and after looking glorious early on, three new spurges are dropping their leaves and looking like they're dying.

When I turn my compost I'm not seeing any worms this year, and have only seen one in the ground when planting. When I take a walk after rain I see lots of slugs on the pavement, but no worms. Anybody else noticed this?