Ryan says he won't do public town halls