RFC: Suggestions for syntax to support encoded password in integration config files

General
,
1

Background

During a recent project I had the opportunity to enhance the way we can treat configuration file values in various (Go based) integrations - YAML in this case but could be other formats such as JSON.

The idea is that you can implied expansions, the current supported set are detailed in ExpandString()

What does this mean? It means you can pull in dynamic values into config files, depending how and when they are used.

An ServiceNow integration (in cordial) does not yet support this, but I will be updating it to work with these changes.

Edit to above: Actually, these changes did get applied to the ServiceNow package, but only in HEAD and not yet in a tagged release.

Help Wanted

Right now the primary missing piece is encoded passwords. For obvious reasons many users don’t like the idea of plaintext password, so we normally reference an external file that is still plaintext but can be better protected against access and accidental viewing.

We already have a Geneos AES256 encoding scheme that uses a key-file to encode/decode “secure” passwords of the form “+encs+XXXX”. We document how to create the key-files and how to encode/decode if you have the key-file. I also have the Go code to support it right now, the core is committed to cordial but more is on the way. You will be able to create encoded key-files and encode password from the geneos command.

My immediate problem is how to put this into a configuration file that:

  • Aligns with the syntax patterns in ExpandString() above
  • Allows the user to implicitly or explicitly refer to a key-file
  • Allows the user to provide the encoded password directly or via any of the styles in ExpandString

Imagine currently either of:

  username: blahblahblah
  password: supersecret

or

  password: ${file:~/.pwfile}

Now, given a path to a key-file (or even two if we want to support the upcoming GA6.0.0 style previous/current system) and an encoded password, perhaps:

  password: ${aes:KEYFILE:ENCPASSWORD}

However, I am really keen - and not just for aesthetic reasons - to avoid supporting any recursive lookups, to here AESFILE and PASSWORD could not themselves use the expansion syntax. Also, if there are two key files, what then?

Ideas, comments, questions please.

Peter

2

First suggestion would be something like:

-  "${enc:keyfile[|keyfile...]:encodedvalue}"
   The item "encodedvalue" references an AES256 cipertext in Geneos
   format which will be decoded using the key files listed. Each
   "keyfile" must be one of either absolute path, a path relative to
   the working directory of the program, or, as for file urls above,
   if prefixed with "~/" then relative to the home directory of the
   user running the program.

   The "encodedvalue" may take the form of any of the other
   references above without the surrounding dollar-brackets
   "${...}". If the encoded value is prefixed with the Geneos style
   "+encs+" then this is automatically removed.

e.g.

  password: ${enc:~/.aeskey.conf:+encs+0B40456637ACEE7AFE81DB9729034BBB}

or

  password: ${enc:~/gw/keyfile.aes|~/gw/prevkeyfile.aes:file:/~/.encoded.pw}

Is this even remotely manageable?

Alternative labels an better English for the doc welcomed!

Note that the management of AES keyfiles, as per Geneos Secure Passwords, is already supported in the HEAD of cordial/tools/geneos and will form part of the next release once I get my head around this small blocker.

3

Second note; In testing I have found that when you try to decode using multiple keyfiles to support rotation there is a chance (I assume 1 in aes blocksize) of the text being decoded but as gibberish.

Is it fair to discard any decoded text that contains characters with non-printables in or are these likely in credentials?

4

What else do you do on a Friday night but watch a James Bond film and have an itch at the back of your head telling you to “just do it” ?

So, this is the result, in doc form. It works and will be checked-in after more testing later this weekend:

5. "${enc:keyfile[|keyfile...]:encodedvalue}"
   The item "encodedvalue" is an AES256 ciphertext in Geneos format
   - or a reference to one - which will be decoded using the key
   file(s) given. Each "keyfile" must be one of either an absolute
   path, a path relative to the working directory of the program, or
   if prefixed with "~/" then relative to the home directory of the
   user running the program. The first valid decode (see below) is
   returned.

   The "encodedvalue" must be either prefixed "+encs+" to align with
   Geneos or will otherwise be looked up using the forms of any of
   the other references above without the surrounding
   dollar-brackets "${...}".

   To minimise (but not eliminate) false decodes in some
   circumstances, if passed the wrong key file, the decoded value is
   only returned if it is a valid UTF-8 string as per [utf8.Valid].

   Examples:

   - password: ${enc:~/.keyfile:+encs+9F2C3871E105EC21E4F0D5A7921A937D}
   - password: ${enc:/etc/geneos/keyfile.aes:env:ENCODED_PASSWORD}

Edit: Except now, after a bad github push and a revert, I screwed up and lost the changes. Luckily I have the above to use as a base to start again :slight_smile:

Edit Edit: Through the power of VSCode local history I got it back. Phew.

5

This looks good, but I feel it would be helpful to run through some scenarios so we can better evaluate them, and to see any which may require additional attention:

  • MAIN file (yaml/json) given to INTERNAL users, for example to replicate environment
    • DIFFERENT servers used
    • SHARED servers used
      • ALL logged on and LOCAL installs, such as all independent teams
        • Local account files must also be used otherwise passwords can be re-used
      • ONE team logged on and MANAGED installs, such as central or shared resource teams
        • Risk exists from misconfiguration (accidents, sabotage, etc) only
  • MAIN file (yaml/json) given to THIRD PARTIES, for example to assist with support queries
    • No risk as cannot be decoded if client key used (but see question below)
  • More scenarios???

One question I had from this, which I don’t see an answer to above, is:
What is meant by “If the encoded value is prefixed with the Geneos style “+encs+” then this is automatically removed.” – is this intended to mean that it uses our own key, rather than a key from a file to do the encryption/decryption, or something else?

Any further thoughts on all the above would be useful, as well as other possibilities. Hope this additional line of thought helps.

Thanks!

6

If you scroll down one reply (to myself) you’ll see the updated doc comment that clarified the use of +encs+.

The risk mitigation here is that you need both the configuration file and the key-file to decode the credentials. There is no way around this without external authentication at start-up. You can pass the configuration around without (or rather with minimal) risk that the encoded credentials can be decoded.

The internal key in the Gateway is not supported and cannot be.

Edit: Existing implementation is now in the repo:

https://pkg.go.dev/github.com/itrs-group/cordial/pkg/config#Config.ExpandString

Peter