Bacillus Thuringiensis, or BT, is a well-known biological control for all types of caterpillars. This organic powder paralyzes the intestinal tract of caterpillars when they begin to feed. Apply BT to the soil around the plants, rather than to leaves, to prevent killing the butterfly larvae you want to encourage. Reapply BT after it rains.
OR
If you’re experiencing a rainy spring that prevents the use of BT, these conditions are optimal for parasitic nematodes. These beneficial nematodes live in the soil and appreciate moist conditions to help them migrate towards their prey. Steinernema carpocapsae and Steinernema feltiae are two commercially available nematodes that control cutworms.