Portable apps with Go and Next.js

· David Stotijn

The release of Go 1.16 introduced a new embed package in the Go standard library. It provides access to files embedded in a Go program at compile time using the new //go:embed directive. It’s a powerful new feature because it allows building a binary with static dependencies like templates, HTML/CSS/JS, or images self-contained. This portability is great for easy distribution and usage. Previously, developers had to rely on third-party libraries for embed behaviour.

In this article we’ll walk through a small demo app, golang-nextjs-portable, that exposes an HTTP server that hosts both an API endpoint, as well as an embedded (Next.js) web app that calls the API.

👉 The source code of the final result is hosted on GitHub.

Go + Next.js = Portable Magic

Embedding a web UI

An interesting use case for the embed package is bundling a web UI in your Go program. A web frontend can have some advantages over a terminal-based UI (TUI). For example, you can run the HTTP server in your home network, and access it from any browser across (mobile) devices. It can also empower you to build user-friendly UI/UX, leveraging web technologies that browsers natively support. When building a hybrid between offline/self-hosted and SaaS, it makes it possible to reuse the same code across self-contained binaries (e.g. for IoT or Raspberry Pi devices) and a hosted/SaaS platform.

Building the Go program

The Go program for our demo project is a relatively simple main.go file:

package main

import (
	"embed"
	"io/fs"
	"log"
	"net/http"
	"runtime/pprof"
)

//go:embed all:nextjs/dist
var nextFS embed.FS

func main() {
	// Root at the `dist` folder generated by the Next.js app.
	distFS, err := fs.Sub(nextFS, "nextjs/dist")
	if err != nil {
		log.Fatal(err)
	}

	// The static Next.js app will be served under `/`.
	http.Handle("/", http.FileServer(http.FS(distFS)))
	// The API will be served under `/api`.
	http.HandleFunc("/api", handleAPI)

	// Start HTTP server at :8080.
	log.Println("Starting HTTP server at http://localhost:8080 ...")
	log.Fatal(http.ListenAndServe(":8080", nil))
}

func handleAPI(w http.ResponseWriter, _ *http.Request) {
	// Gather memory allocations profile.
	profile := pprof.Lookup("allocs")

	// Write profile (human readable, via debug: 1) to HTTP response.
	err := profile.WriteTo(w, 1)
	if err != nil {
		log.Printf("Error: Failed to write allocs profile: %v", err)
	}
}

It uses the http package to run a simple HTTP web server on port :8080. The API is served under /api and returns a text response with current memory allocation stats.

Using the embed package

To embed files into our Go program, we’ll use the standard library embed package, introduced in Go 1.16. See the package docs for details.

We’ll use the //go:embed directive like so:

//go:embed all:nextjs/dist
var nextFS embed.FS

The directory nextjs/dist will contain the static HTML export of the Next.js app (see “The export script” in the next section).

The all: prefix (added in Go 1.18) ensures that any files or directories prefixed with . or _ are included:

If a pattern begins with the prefix ‘all:’, then the rule for walking directories is changed to include those files beginning with ‘.’ or ‘_’. For example, ‘all:image’ embeds both ‘image/.tempfile’ and ‘image/dir/.tempfile’.

(Source: embed package docs)

Because a static HTML export of a Next.js app contains various directories and files prefixed with _, we need this rule behavior.

At compile time, the matched files are embedded in the binary. The embed.FS type implements fs.FS, which was also introduced in Go 1.16. Because we want to serve the static HTML export at /, we’ll use fs.Sub. This method returns a fs.FS value that (under the hood) maps to the subtree nextjs/dist).

// Root at the `dist` folder generated by the Next.js app.
distFS, err := fs.Sub(nextFS, "nextjs/dist")
if err != nil {
	log.Fatal(err)
}

// The static Next.js app will be served under `/`.
http.Handle("/", http.FileServer(http.FS(distFS)))

Building the Next.js app

For this article, I’m going to skip setting up and scaffolding a Next.js app. We’ll only cover the relevant code and configuration for our embedding purposes:

Making API requests

The homepage of our app will fetch memory allocation stats from the (Go) HTTP server, which exposes an /api route. We’ll use swr, a lightweight React Hook for fetching data. We fetch using only the path, which will cause the request to made to the server that’s rendering the current HTML page. This is convenient because fetching will work both when running Next.js in development (using its own Node.js web server, by default on :3000) as well as via the Go HTTP server, covered in the next section.

import Link from "next/link";
import useSWR from "swr";

async function fetcher(url: string) {
  const resp = await fetch(url);
  return resp.text();
}

function Index(): JSX.Element {
  const { data, error } = useSWR("/api", fetcher, { refreshInterval: 1000 });

  return (
    <div>
      <h1>Hello, world!</h1>
      <p>This is <code>pages/index.tsx</code>.</p>
      <p>Check out <Link href="/foo">foo</Link>.</p>

      <h2>Memory allocation stats from Go server</h2>
      {error && <p>Error fetching profile: <strong>{error}</strong></p>)}
      {!error && !data && <p>Loading ...</p>}
      {!error && data && <pre>{data}</pre>}
    </div>
  );
}

export default Index;

Proxying API requests in dev mode

When running Next.js in development via next dev, it spawns a Node.js server serving the app (with live reloading) at :3000. When our fetch code runs, it will make a GET request to http://localhost:3000/api. Obviously, this won’t work out of the box; the Next.js dev server will attempt to serve pages/api.tsx, which doesn’t exist. What we want is requests to /api to be proxied to our Go server, which we’ll run on :8080. That way we don’t have to constantly do a manual export every time we make a change to our Next.js code.

The proxy configuration can be defined in nextjs/next.config.js:

module.exports = {
  async rewrites() {
    // When running Next.js via Node.js (e.g. `dev` mode), proxy API requests
    // to the Go server.
    return [
      {
        source: "/api",
        destination: "http://localhost:8080/api",
      },
    ];
  },
};

Note: Changes to next.config.js only take effect when restarting the Next.js dev server.

The steps for running Next.js in dev mode (with live reloading) are:

  1. First, create an export of the Next.js app using yarn run export. We need to do this first/once because the Go program will only compile if there are files to embed.
  2. Run the Next.js dev server: yarn run dev.
  3. Build/run the Go program, using go run main.go.
  4. Access the app via http://localhost:3000. The API requests to /api should be proxied to the Go server running on :8080.

The export script

Next.js has first class support for static HTML export. Typically it’s used to deploy on a static hosting service or CDN. In our use case, we’ll embed the export in a Go program.

By default, running next export will write to an out directory. I prefer the name dist. To prevent unneeded bundle contents for repeated exports after updating JS imports, we’ll want to remove the .next directory before generating a static export. We an export npm script as follows:

{
  (...)
  "scripts": {
    (...)
    "export": "rm -rf .next && next build && next export -o dist"
  }
}

Now, running yarn run export will result in a static HTML export written to a nextjs/dist directory. Note: Next.js recreates the export directory each time when repeatedly exporting, so there’s no need to manually remove it.

Build & distribute

After adding both the Go program and the Next.js app to our code repository, we end up with the following directory structure.

./golang-nextjs-portable
├── go.mod
├── main.go               <- Go program
└── nextjs
    ├── dist              <- Export directory created with `yarn run export`
    ├── next-env.d.ts
    ├── next.config.js
    ├── package.json
    ├── package.json
    ├── pages             <- Next.js pages
    │   ├── foo
    │   │   ├── bar.tsx
    │   │   └── index.tsx
    │   └── index.tsx
    ├── tsconfig.json
    ├── tsconfig.json
    └── yarn.lock

Building

Building consists of two steps: generating the static HTML export, and building the Go program:

$ cd nextjs
$ yarn run export
$ cd ..
$ go build main.go

This generates an executable golang-nextjs-portable file.

Running

To run the program, we simply execute the binary:

$ ./golang-nextjs-portable

2021/04/27 14:55:38 Starting HTTP server at http://localhost:8080 ...

When you open the web interface via http://localhost:8080, you should see the statically generated HTML page, with live memory allocation stats coming from our API, refreshed every second:

Static HTML export calling API

Dockerfile

To create a streamlined Docker image for our portable app, we create a Dockerfile:

ARG GO_VERSION=1.18
ARG NODE_VERSION=14.16.1
ARG ALPINE_VERSION=3.13.5

FROM node:${NODE_VERSION}-alpine AS node-builder
WORKDIR /app
COPY nextjs/package.json nextjs/yarn.lock ./
RUN yarn install --frozen-lockfile
COPY nextjs/ .
ENV NEXT_TELEMETRY_DISABLED=1
RUN yarn run export

FROM golang:${GO_VERSION}-alpine AS go-builder
WORKDIR /app
COPY go.mod main.go ./
COPY --from=node-builder /app/dist ./nextjs/dist
RUN go build .

FROM alpine:${ALPINE_VERSION}
WORKDIR /app
COPY --from=go-builder /app/golang-nextjs-portable .

ENTRYPOINT ["./golang-nextjs-portable"]

EXPOSE 8080

To keep the final image size down, we use multi-stage builds. The total image size of this demo app is 13MB, including all resources needed to run the app. Not bad!

Closing thoughts

I’ve used the approach outlined in this article for Hetty, a project that uses Go for its core behaviour, and Next.js for the web interface. For me, this offers the best of both worlds: using Go for general-purpose programming and its amazing standard library, and HTML/JS/CSS with its ever evolving ecosystem of UI and UX tooling. Be mindful of the cohesion between server and client though. Using some contract between the two (GraphQL, protobuf, JSON schema) is advised for larger/real-world projects.

On the other hand, bundling a web app can be complete overkill if your Go program doesn’t have elaborate UI/UX needs. As with almost everything: “it depends”.

Do you have any corrections/suggestions? Please get in touch!