if you like to feed a lot, just like i do, i would put the skimmer back. specially with not so regular water changes, they do like more nutrients in the water then most SPS corals but still don't like a lot of nitrates which it seems you are heading in that direction if you don't stop being lazy
of course there are skimmer less tanks but im sure they keep up with water changes and feed sparingly.