If you care even a little about your public image, then refrain completely from public display of affection especially in office. It will help if you can save the handholding and the caressing stuff for the hours after both of you have exited from office. Infact, such candid display of affection in office will be interpreted as being unprofessional in any office. Also avoid sending love messages to one another in office. You will become the butt of everyone’s jokes if your lovey-dovey emails get viewed by someone else accidentally in the office.