# Chapter 5. The FreeBSD Documentation Build Process

This chapter covers organization of the documentation build process and how make(1) is used to control it.

## 5.1. Rendering AsciiDoc into Output

Different types of output can be produced from a single AsciiDoc source file.

Formats | File Type | Description
--- | --- | ---
`html` | HTML | An `article` or `book` chapter.
`pdf` | PDF | Portable Document Format.
`epub` | EPUB | Electronic Publication. ePub file format.

### 5.1.1. Rendering to html

To render the documentation and the website to `html` use one of the following examples.

Example 1. Build the documentation

    % cd ~/doc/documentation
    % make

Example 2. Build the website

    % cd ~/doc/website
    % make

Example 3. Build the entire documentation project

    % cd ~/doc
    % make -j2

Advanced build examples are given below:

Example 4. Build English and Spanish documentation with verbose and debug messages

    % cd ~/doc/documentation
    % make DOC_LANG="en es" HUGO_ARGS="--verbose --debug"

Example 5. Build and serve the content with Hugo’s internal webserver

    % cd ~/doc/documentation
    % make run

This webserver runs on `localhost`, port `1313` by default.

To serve the content with Hugo’s internal webserver binding a specific IP address and port:

    % make run BIND=192.168.15.10 HUGO_ARGS="-p 8080"

A `hostname` can also be set as base url to Hugo’s internal webserver:

    % make run BIND=192.168.15.10 HOSTNAME=example.com

Example 6. Build documentation in html for offline usage

    % cd ~/doc/documentation
    % make html

To compress the html output, add `DOC_HTML_ARCHIVE=1`:

    % cd ~/doc/documentation
    % DOC_HTML_ARCHIVE=1 make html

### 5.1.2. Rendering to pdf

To render the documentation to `pdf`, use one of the following examples.

Example 7. Build all documents in pdf

    % cd ~/doc/documentation
    % make pdf

Example 8. Build all articles in pdf

    % cd ~/doc/documentation
    % make pdf-articles

Example 9. Build all books in pdf

    % cd ~/doc/documentation
    % make pdf-books

Example 10. Build documents in pdf for specific languages

    % cd ~/doc/documentation
    % make DOC_LANG="en" pdf

This will build all English documents in pdf.

    % cd ~/doc/documentation
    % make DOC_LANG="en fr" pdf-books

This will build all English and French books in pdf.

## 5.2. The FreeBSD Documentation Build Toolset

These are the tools used to build and install the FDP documentation.

- The primary build tool is make(1), specifically Berkeley Make.

- Hugo

- AsciiDoctor

- Git

## 5.3. Understanding the Makefile in the Documentation Tree

There are three Makefile files for building some or all of the documentation project.

- The Makefile in the documentation directory will build only the documentation.

- The Makefile in the website directory will build only the website.

- The Makefile at the top of the tree will build both the documentation and the website.

The Makefile appearing in subdirectories also support `make run` to serve built content with Hugo’s internal webserver. This webserver runs on port 1313 by default.

### 5.3.1. Documentation Makefile

This Makefile takes the following form:

    # Generate the FreeBSD documentation
    #
    # Copyright (c) 2020-2021, The FreeBSD Documentation Project
    # Copyright (c) 2020-2021, Sergio Carlavilla <carlavilla@FreeBSD.org>
    #
    # Targets intended for use on the command line
    #
    # all (default)	-	generate the books TOC and compile all the documentation
    # clean		- 	removes generated files
    # run		-	serves the built documentation site for local browsing
    # pdf		-	build PDF versions of the articles and books.
    # html		-	build HTML versions of the articles and books for
    #			offline use.
    #			If variable DOC_HTML_ARCHIVE is set, all documents will be
    #			archived/compressed, and only these files will be kept in the public
    #			directory.
    # epub		-	build EPUB versions of the articles and books (Experimental).
    #
    # The run target uses hugo's built-in webserver to make the documentation site
    # available for local browsing.  The documentation should have been built prior
    # to attempting to use the `run` target.  By default, hugo will start its
    # webserver on port 1313.

    MAINTAINER=carlavilla@FreeBSD.org (1)

    # List of languages without book translations
    ARTICLEONLY_LANGS=	bn-bd da ko tr
    # List of languages without article translations
    BOOKONLY_LANGS=		mn

    # List of all languages we have content for
    ALL_LANGUAGES=	bn-bd da de el en es fr hu it ja ko mn nl pl pt-br ru tr zh-cn zh-tw (2)

    LOCALBASE?=	/usr/local

    RUBY_CMD =	${LOCALBASE}/bin/ruby (3)
    HUGO_CMD =	${LOCALBASE}/bin/hugo (4)
    HUGO_ARGS?=	--verbose --minify
    HUGO_OFFLINE_ARGS?= 	--environment offline --verbose --minify
    ASCIIDOCTOR_CMD=	${LOCALBASE}/bin/asciidoctor
    ASCIIDOCTORPDF_CMD=	${LOCALBASE}/bin/asciidoctor-pdf

    .if defined(DOC_LANG) && !empty(DOC_LANG)
    LANGUAGES=	${DOC_LANG:S/,/ /g}
    .if  ${LANGUAGES:Men} == "" && ${.TARGETS:Mpdf*} == "" && ${.TARGETS:Mhtml*} == ""
    .warning "Warning: cannot skip 'en'; adding it back"
    LANGUAGES+=	en
    .endif
    .else
    LANGUAGES=	${ALL_LANGUAGES}
    .endif

    RUBYLIB =	../shared/lib
    .export	RUBYLIB

    RUN_DEPENDS=	${HUGO_CMD} \
    		${LOCALBASE}/bin/asciidoctor \
    		${LOCALBASE}/bin/rougify

    .ifndef HOSTNAME
    .  ifdef BIND
    .HOST=$(BIND)
    .  else
    .HOST=localhost
    .  endif
    .else
    .HOST=$(HOSTNAME)
    .endif

    # Strip the languages with only articles from the list of languages we
    #  will use to build books.
    BOOK_LANGS= ${LANGUAGES}
    .for a in ${ARTICLEONLY_LANGS}
    BOOK_LANGS:=	${BOOK_LANGS:N${a}}
    .endfor

    # Strip the languages with only books from the list of languages we
    #  will use to build articles.
    ARTICLE_LANGS= ${LANGUAGES}
    .for a in ${BOOKONLY_LANGS}
    ARTICLE_LANGS:=	${ARTICLE_LANGS:N${a}}
    .endfor

    # Take the list of all languages, and take out the ones we have been
    #   asked for.  We'll feed this to hugo.
    SKIP_LANGS=
    .for a in ${ALL_LANGUAGES}
    .if  ${LANGUAGES:M${a}} == ""
    SKIP_LANGS+=    ${a}
    .endif
    .endfor

    .ORDER: all run (5)

    .ORDER: requirements (6)
    .ORDER: starting-message
    .ORDER: starting-message build
    .ORDER: build

    all: requirements starting-message generate-pgpkeys-txt build
    run: requirements starting-message generate-pgpkeys-txt run-local

    # clean does not call pdf-clean as that is a subset of hugo-clean
    clean: hugo-clean pgp-clean

    requirements:
    .for dep in ${RUN_DEPENDS}
    .if !exists(${dep})
    	@(echo ${dep} not found, please run 'pkg install docproj'; exit 1)
    .endif
    .endfor

    requirements-pdf:
    .if !exists(${LOCALBASE}/bin/asciidoctor-pdf)
    	@(echo ${LOCALBASE}/bin/asciidoctor-pdf not found, please run 'pkg install rubygem-asciidoctor-pdf'; exit 1)
    .endif

    requirements-epub:
    .if !exists(${LOCALBASE}/bin/asciidoctor-epub3)
    	@(echo ${LOCALBASE}/bin/asciidoctor-epub3 not found, please run 'pkg install rubygem-asciidoctor-epub3'; exit 1)
    .endif

    starting-message: .PHONY (7)
    	@echo ---------------------------------------------------------------
    	@echo                   Building the documentation
    	@echo  included languages: ${LANGUAGES}
    	@echo  excluded languages: ${SKIP_LANGS}
    	@echo ---------------------------------------------------------------

    generate-pgpkeys-txt: static/pgpkeys/pgpkeys.txt

    static/pgpkeys/pgpkeys.txt: static/pgpkeys/*key
    	${RUBY_CMD} ./tools/global-pgpkeys-creator.rb

    run-local: .PHONY (8)
    	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} server \
    		${HUGO_ARGS} -D $(BIND:D--bind=$(BIND)) --baseURL="http://$(.HOST):1313"

    build: .PHONY (9)
    	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_ARGS}

    build-offline: .PHONY
    	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_OFFLINE_ARGS}

    pgp-clean: .PHONY
    	rm -f static/pgpkeys/pgpkeys.txt

    hugo-clean: .PHONY
    	rm -rf resources public

    #
    # PDF targets
    # Use DOC_LANG to choose the language, e.g., make DOC_LANG="en fr" pdf-books
    #
    pdf: pdf-articles pdf-books

    pdf-books: requirements-pdf
    .for _lang in ${BOOK_LANGS}
    	./tools/asciidoctor.sh books ${_lang} pdf
    .endfor

    pdf-articles: requirements-pdf
    .for _lang in ${ARTICLE_LANGS}
    	./tools/asciidoctor.sh articles ${_lang} pdf
    .endfor

    pdf-clean: pdf-articles-clean pdf-books-clean

    pdf-books-clean:
    .for _lang in ${BOOK_LANGS}
    	rm -fr ${.CURDIR}/public/${_lang}/books
    	-rmdir ${.CURDIR}/public/${_lang}
    .endfor
    	-rmdir ${.CURDIR}/public/

    pdf-articles-clean:
    .for _lang in ${ARTICLE_LANGS}
    	rm -fr ${.CURDIR}/public/${_lang}/articles
    .if !exists(${.CURDIR}/public/${_lang}/books)
    	rm -fr ${.CURDIR}/public/${_lang}
    .endif
    .endfor
    	-rmdir ${.CURDIR}/public

    #
    # HTML targets
    #
    html: build-offline html-clean-global html-clean-articles html-clean-books html-archive html-archive-clean-files

    html-clean: hugo-clean

    html-clean-global:
    	rm -fr ${.CURDIR}/public/index.html
    	rm -rf pgpkeys js

    html-clean-articles:
    .for _lang in ${ARTICLE_LANGS}
    	rm -fr ${.CURDIR}/public/${_lang}/index.html
    	rm -fr ${.CURDIR}/public/${_lang}/articles/index.html
    .endfor

    html-clean-books:
    .for _lang in ${BOOK_LANGS}
    	rm -fr ${.CURDIR}/public/${_lang}/books/index.html
    .endfor

    html-archive:
    .if defined(DOC_HTML_ARCHIVE)
    .for _lang in ${ARTICLE_LANGS}
    	./tools/asciidoctor.sh articles ${_lang} archive
    .endfor
    .for _lang in ${BOOK_LANGS}
    	./tools/asciidoctor.sh books ${_lang} archive
    .endfor
    .endif

    html-archive-clean-files:
    .if defined(DOC_HTML_ARCHIVE)
    	find ${.CURDIR}/public/ ! -name '*.pdf' ! -name '*.tar.gz' -type f -delete
    	find ${.CURDIR}/public/ -type d -empty -delete
    .endif

    #
    # EPUB targets
    # Use DOC_LANG to choose the language, e.g., make DOC_LANG="en fr" epub-books
    #
    epub: epub-articles epub-books

    epub-books: requirements-epub
    	@echo ---------------------------------------------------------------
    	@echo !!! EPUB output is experimental !!!
    	@echo
    	@echo Asciidoctor EPUB3 is currently alpha software. Use accordingly. Although the
    	@echo bulk of AsciiDoc content is converted, there’s still work needed to fill in
    	@echo gaps where conversion is incomplete or unstyled.
    	@echo https://docs.asciidoctor.org/epub3-converter/latest/#project-status
    	@echo ---------------------------------------------------------------
    .for _lang in ${BOOK_LANGS}
    	./tools/asciidoctor.sh books ${_lang} epub
    .endfor

    epub-articles: requirements-epub
    	@echo ---------------------------------------------------------------
    	@echo !!! EPUB output is experimental !!!
    	@echo
    	@echo Asciidoctor EPUB3 is currently alpha software. Use accordingly. Although the
    	@echo bulk of AsciiDoc content is converted, there’s still work needed to fill in
    	@echo gaps where conversion is incomplete or unstyled.
    	@echo https://docs.asciidoctor.org/epub3-converter/latest/#project-status
    	@echo ---------------------------------------------------------------
    .for _lang in ${ARTICLE_LANGS}
    	./tools/asciidoctor.sh articles ${_lang} epub
    .endfor

    epub-clean: epub-articles-clean epub-books-clean

    epub-books-clean:
    .for _lang in ${BOOK_LANGS}
    	rm -fr ${.CURDIR}/public/${_lang}/books
    	-rmdir ${.CURDIR}/public/${_lang}
    .endfor
    	-rmdir ${.CURDIR}/public/

    epub-articles-clean:
    .for _lang in ${ARTICLE_LANGS}
    	rm -fr ${.CURDIR}/public/${_lang}/articles
    .if !exists(${.CURDIR}/public/${_lang}/books)
    	rm -fr ${.CURDIR}/public/${_lang}
    .endif
    .endfor
    	-rmdir ${.CURDIR}/public

1 | The `MAINTAINER` flag specifies who is the maintainer of this Makefile.
2 | `ALL_LANGUAGES` flag specifies in which languages the table of contents has to be generated.
3 | `RUBY_CMD` flag specifies the location of the Ruby binary.
4 | `HUGO_CMD` flag specifies the location of the Hugo binary.
5 | `.ORDER` directives are used to ensure multiple make jobs may run without problem.
6 | `all` target builds the documentation and puts the result in ~/doc/documentation/public.
7 | `starting-message` shows a message in the CLI to show the user that the process is running.
8 | `run-local` runs hugo webserver on port 1313, or a random free port if that is already in use.
9 | `build` builds the documentation and puts the result in the ~/doc/documentation/public.

### 5.3.2. Website Makefile

This Makefile takes the form of:

    # Generate the FreeBSD website
    #
    # Copyright (c) 2020-2021, The FreeBSD Documentation Project
    # Copyright (c) 2020-2021, Sergio Carlavilla <carlavilla@FreeBSD.org>
    #
    # Targets intended for use on the command line
    #
    # all (default)	-	generate the releases.toml and compile all the website
    # run	-			serves the built website for local browsing
    #
    # The run target uses hugo's built-in webserver to make the built website
    # available for local browsing.  The website should have been built prior
    # to attempting to use the `run` target.  By default, hugo will start its
    # webserver on port 1313.

    MAINTAINER=carlavilla@FreeBSD.org (1)

    # List of all languages we have content for
    ALL_LANGUAGES=	de el en es fr hu it ja nl ru tr zh-cn zh-tw

    LOCALBASE?=	/usr/local

    RUBY_CMD =	${LOCALBASE}/bin/ruby (2)
    HUGO_CMD =	${LOCALBASE}/bin/hugo (3)
    HUGO_ARGS?=	--verbose
    RUBYLIB =	../shared/lib
    .export	RUBYLIB

    .ifndef HOSTNAME
    .  ifdef BIND
    .HOST=$(BIND)
    .  else
    .HOST=localhost
    .  endif
    .else
    .HOST=$(HOSTNAME)
    .endif

    .if defined(DOC_LANG) && !empty(DOC_LANG)
    LANGUAGES=      ${DOC_LANG:S/,/ /g}
    .if  ${LANGUAGES:Men} == ""
    .warning "Warning: cannot skip 'en'; adding it back"
    LANGUAGES+=	en
    .endif
    .else
    LANGUAGES=	${ALL_LANGUAGES}
    .endif

    # Take the list of all languages, and take out the ones we have been
    #   asked for via DOC_LANG.  We'll feed this to hugo.
    SKIP_LANGS=
    .for a in ${ALL_LANGUAGES}
    .if ${LANGUAGES:M${a}} == ""
    SKIP_LANGS+=	${a}
    .endif
    .endfor

    .ORDER: all run (4)

    .ORDER: starting-message generate-releases
    .ORDER: starting-message build
    .ORDER: generate-releases build
    .ORDER: build post-build
    .ORDER: post-build end-message

    all: starting-message generate-releases build post-build end-message (5)
    run: starting-message generate-releases run-local
    clean: hugo-clean releases-clean

    starting-message: .PHONY (6)
    	@echo "---------------------------------------------------------------"
    	@echo "Building the website started on $$(date)"
    	@echo " included languages: ${LANGUAGES}"
    	@echo " excluded languages: ${SKIP_LANGS}"
    	@echo "---------------------------------------------------------------"

    end-message: .PHONY
    	@echo "---------------------------------------------------------------"
    	@echo "Building the website completed on $$(date)"
    	@echo "---------------------------------------------------------------"

    generate-releases: data/releases.toml (7)

    data/releases.toml:
    	${RUBY_CMD} ./tools/releases-toml.rb

    run-local: .PHONY (8)
    	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} server \
    	    ${HUGO_ARGS} -D $(BIND:D--bind=$(BIND)) --baseURL="http://$(.HOST):1313"

    build: .PHONY (9)
    	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_ARGS}

    post-build: cgi-permissions

    cgi-permissions:
    	@chmod 555 ./public/cgi/*.cgi

    hugo-clean:
    	rm -fr public resources

    releases-clean:
    	rm -f data/releases.toml

1 | The `MAINTAINER` flag specifies who is the maintainer of this Makefile.
2 | `RUBY_CMD` flag specifies the location of the Ruby binary.
3 | `HUGO_CMD` flag specifies the location of the Hugo binary.
4 | `.ORDER` directives are used to ensure multiple make jobs may run without problem.
5 | `all` target builds the website and puts the result in ~/doc/website/public.
6 | `starting-message` shows a message in the CLI to show the user that the process is running.
7 | `generate-releases` calls the script used to convert from AsciiDoc variables to TOML variables. With this conversion, the releases variables can be used in AsciiDoc and in the Hugo custom templates.
8 | `run-local` runs hugo webserver on port 1313, or a random free port if that is already in use.
9 | `build` builds the website and puts the result in the ~/doc/website/public.

Last modified on: March 9, 2024 by Danilo G. Baio

Prev
Home
Next