WDYM the JS way?

I mean the “dynamic languages” way. For example, Pydantic Settings and the config system in Spotify’s Backstage.

This new release (v0.2) is all metagprogramming at runtime with Go’s reflect package. This is a 180° from my previous version, but hear me out!

It’s really good! Yeah, I’m biased, of course. Hit me up on email/socials if you think I did something messed up here :)

What is factor3?

factor3 is an open source Go library I’m writing. It’s a way to abstract and simplify working with the concept of a “config”. In a very hand-wavy way, let’s define config to be the inputs you receive to the program from some place that is not the main way your program interacts with the world.

Every Go programmer that has written a CLI most likely have heard of spf13/cobra, and maybe also about spf13/viper. These are two highly customizable framewroks for parsing cli flags, loading in env variables and reading from a config file.

I wanted to build on top of these tools and enable additional workflows that also fall under the category of “config”.

For example, when writing a web server, our main input method is through a tcp socket. Other inputs are, for example

• Environment and CLI arguments
• User defined config.json or yaml or toml
• Raw files that are not in json format (e.g pem files)
• Pull sensitive secrets from some secret storage app (e.g 1Password, AWS Secret Manager, K8s Secrets)
• Feature flags set is some SaaS platform outside of the cluster
• Annotations set on the K8s Pod running this container (e.g for rolling updates via Argo Rollouts)

Piecing together your inputs from so many types of systems becomes a whole project out if itself. It’s always NOT what you want to spend time on when starting a new Go project. For me, it’s always a big distraction from the prototype I’m trying to build. I always end up doing it the same sloppy manual glue code for at least having some basic cobra+viper app.

factor3 is my attempt at streamlining all of these difference sources of input into a unified interface that can be just globally consumed from everywhere in your go app.

It’s not about hiding to complexity behind these systems. It’s about separation of concerns between collecting the information from multiple sources versus using the values to drive your app’s behavior.

How it feels

; go get github.com/drornir/factor3

And here’s a small program that actually works, copy paste it and try it yourself!

Check out the comments for extra explanation

package main

import (
	"fmt"
	"os"

	"github.com/drornir/factor3/pkg/factor3"
	"github.com/fsnotify/fsnotify"
	"github.com/spf13/cobra"
	"github.com/spf13/viper"
)

// Define the struct you want to factor3 to work with
type (
	Config struct {
		Username string               `flag:"username" json:"username"`
		Password factor3.SecretString `flag:"password" json:"password"`
		Log      LogConfig            `flag:"log" json:"log"`
	}
	LogConfig struct {
		Level string `flag:"level" json:"level"`
	}
)

var (
	// define a variable to bind with factor3.Bind()
	rootConfig Config
	// an example cobra command that uses the config
	rootCmd    = &cobra.Command{
		Use: "myprogram",
		Run: func(cmd *cobra.Command, args []string) {
			fmt.Printf("# config = %#v\n", rootConfig)
		},
	}
)

func init() {
	viperInstance := viper.New()

	// Setting up viper with options that fit factor3
	err := factor3.InitializeViper(factor3.InitArgs{
		Viper:       viperInstance,
		ProgramName: "myprogram", // Used as the env variables prefix
		CfgFile:     "config.json", // Optional path to config file
	})
	cobra.CheckErr(err)

	pflags := rootCmd.Flags()
	// Using Bind() we create Loader that populates the config when called
	// It also registers the flags in your pflag.FlagSet
	loader, err := factor3.Bind(&rootConfig, viperInstance, pflags)
	cobra.CheckErr(err)

	// we need to let cobra parse to commandline flags before calling Load(), so we put it in cobra.OnInitialize()
	cobra.OnInitialize(func() {
		err := loader.Load()
		cobra.CheckErr(err)
		// Advanced: You can call Load() multiple times, for example in reaction to changes to the config file.
		viperInstance.OnConfigChange(func(in fsnotify.Event) {
			if err := loader.Load(); err != nil {
				fmt.Println("error reloading config on viper.OnConfigChange")
			}
		})
	})
}

func main() {
	err := rootCmd.Execute()
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

To test it out, create a simple config.json file next to your main.go and experiment!

Here’s a short example:

cat <<EOF > config.json
{
  "username": "u",
  "password": "p",
  "log": {
    "level": "warn"
  }
}
EOF

## reads the file specified by `CfgFile`,
; go run main.go
# config = main.Config{Username:"u", Password:"p", ...}

## flags that were explicitly set on the command line will override the config file
; go run main.go --username=u_flag --password=p_flag
# config = main.Config{Username:"u_flag", Password:"p_flag", ...}

## env vars that were explicitly set will override the config file
## note that "MYPROGRAM_" comes from the ProgramName set in the `InitArgs`
; MYPROGRAM_PASSWORD='p_env' go run main.go --username=u_flag
# config = main.Config{Username:"u_flag", Password:"p_env", ...}

## nested fields can be set using underscore ('_') for ENV, and dash ('-') for flags
; MYPROGRAM_LOG_LEVEL=info go run main.go --log-level=debug
# config = main.Config{..., Log:main.LogConfig{Level:"debug"}}

Why did I rewrite?

I didn’t like the previous version of factor3. It was hard to understand what are the right abstractions to give to the user. I was stuck.

So I decided to try again, now starting from the “other side” - instead of starting with code generation, I’ll create something using Go’s standard reflect package and only later I’ll convert some of the functionality to be code generated instead of computed as runtime.

I understood that I tried to do step two of this project before doing step one - make it work.

There’s still much to do, but the current version is usable for me on another project I’m starting so it made sense to me to make a release.

Where did all the code generation go?

It will be back! Stay tuned!

If you’ve read so far, make sure to star and watch te repo for new releases! 😉

github.com/drornir/factor3

---

Cover photo by Simon Williams on Unsplash