While it might at first seem a bit strange why would you need an office suite like LibreOffice on a headless system, there's at least one good reason for it - the ability to convert between different formats. In fact, both OpenOffice.org and LibreOffice explicitly support headless mode.
A headless system with FreeBSD will likely have
/etc/make.conf (note that it doesn't matter whether the variable is set to 'yes', 'true', '1' or even 'no', 'false' or '0' - what counts is whether it is set at all or not; the contents are irrelevant). This should make some ports automagically disable X11 bindings or other stuff that relies on X11.
LibreOffice builds well with this variable defined, with one important exception, however: the graphics/cairo port. Cairo itself build without issues, too, but when you get to building LibreOffice, it will complain about missing cairo-xlib. And the problem here is that cairo-xlib is not built when WITHOUT_X11 is defined.
Overcoming this is actually easy (perhaps there's even more elegant solution, but I haven't had the time to investigate further). Just comment out WITHOUT_X11 in make.conf temporarily, (force) upgrade Cairo, then uncomment it back, and finally build LibreOffice. This will work with subsequent updates to LO as well; it's only necessary to comment WITHOUT_X11 when (and only while) upgrading Cairo.