Yes you do.  You, your fiancé and a witness must make an appointment to come in.

https://saddlebrooknj.us/health-department-and-registrar/home/