I think you can in a knit but in a woven, I would not omit the zip. The front opening is more decorative and the neckline fits pretty high.