I try to buy organic as much as possible. By shopping sales, I find organic produce for less than regular produce -- or at least the same price.
Along with my shopping list, I carry a list of items the "experts" suggest should be a priority in getting organic.
So, if the market does not have organic, say, watermelon, but they do have regular ones on sale, then I don't feel too badly about purchasing it.
(The organic priority list varies from website to website, but this is the particular list I keep with me.)