RELEASE: Cordial v1.3.0 - Tools to manage and interact with Geneos

We are very happy to announce this release of cordial v1.3.0.

This version adds a number of new features and fixes a plethora of bugs.

Please let us know of any issues or new features you want to see.

Change Log

Version v1.3.0 - 2022/10/25

Changes

  • PagerDuty integration
  • Merged ServiceNow integration, single binary build
  • tools/geneos: add instance protection against stop (and related) or delete commands
  • tools/geneos: support legacy command through emulating *ctl named commands
  • tools/geneos: allow remote operations without local directories

Fixes

  • tools/geneos: fix logic around creating user default AES keyfiles and directory permissions
  • tools/geneos: round certificate expiry to midnight
  • tools/geneos: round tls remaining column to seconds correctly
  • tools/geneos: fix webserver command build typo. now webserver starts correctly
  • libemail: fix default _SUBJECT handling
  • tools/geneos: split over complex ‘init’ command into sub-commands
  • updated command usage information and reordered various internal function calls
  • tools/geneos: add password verify to aes encode and a --once flag to override
  • tools/geneos: add local JoinSlash and Dir to use Linux paths on Windows builds
  • tools/geneos: fix ssh-agent support on windows
  • tools/geneos: build on windows
  • integrations: Add PagerDuty integration
  • Integrations: Merge ServiceNow binaries into one
  • tools/geneos: change internal remote Stat() API

Notes for previous release, v1.2.1

There have been many changes since the original v1.0.0 release in June 2022. Please see the CHANGELOG.md file for more information.

The highlights include:

  • Lots of geneos program improvements, including numerous bug fixes and new features like AES encoding support
  • The pkgs get many improvements, the XML-RPC packages have been refactored to work in a real-world application
    • New config package to add a variety of features over the top of viper
    • New geneos package to aid in the construction of XML configurations for Gateway and Netprobe. This is work in progress.
    • New commands package to provide the start of support for REST API Commands to the Gateway. This is work in progress.
    • New xpath package to work with the above and also the base for the snapshot command below. This is also work in progress.
    • New cordial package that initially carries a version constant and a general purpose LogInit() function.
    • New process package, providing a way to Daemon()ise callers on both Linux and Windows.
  • The Servicenow integration has clean-up work under the hood, adding better configuration file support using the new package(s) above
  • Use of zerolog for logging in most cases throughout. The existing logger package will be deprecated in the next release

Individual binaries for Linux/AMD64 and example configurations can be found attached to the release, as well as tar.gz and zip files containing the same files for convenience.

Work continues and we appreciate any feedback, suggestions or bug reports you have. For bugs please either raise an Issue in github or, for more general things, post here.

We welcome feedback and contributions.


cordial

Cordial comprises a set of interrelated programs, integrations, libraries and packages for Geneos, principally written in Go.

  • geneos Manager Program
  • ServiceNow Incident Integration
  • Enhanced libemail
  • Go Packages
    • commands
      Geneos Gateway REST API Commands
    • config
      Configuration file support, based on viper with local extensions
    • geneos
      Automated Geneos XML configuration file generations based on Go data structures
    • process
      Process management utilities.
    • plugins, samplers, streams, xmlrpc
      Geneos API plugin XML-RPC support
    • xpath
      Geneos XPath handling

Version v1.2.1-rc3 - 2022/10/07

Fixes

  • geneos command fixes:

    • Fixed init download credential handling
    • Fixes JSON output formay from ls commands
    • Local-only installs now work again (including default “latest” support)
  • Security

    • Updated Labstack Echo to 4.9.0 to address security advisory CVE-2022-40083. To best of our knowledge this particular set of features was never used in this package.

Additional features and improvements

  • geneos command improvements:

    • Added --raw to show to not output expanded configuration values
    • Many improvements and changes to the new aes sub-commands. Please see documentation for details
    • Removed built-in opaquing of credentials in output in favour of new ${enc:...} support
  • libemail.so gets direct passwords back, with ExpandString support. See documentation for details

  • General package improvements

    • Enhanced OpenLocalFileOrURL to support ~/ paths
    • Enhanced ExpandString to support direct file paths and updates package docs further

Nice. I am using geneos tool since 2013. Its awesome. It will be great if this is available in reasonable price in order to compete other available monitoring tools in the market.

Hi Pinku, thanks for the positive comment. This specific tool set I have announced is not Geneos itself but tool(s) to help manage large, multi-component deployments that should improve ease of use over existing shell scripts etc.

Great news and add to the forum, thanks a lot @pgalbavy :+1:

Hello Peter,

It sounds great. We would need to use the SECURE/TLS extension of SMTP and ITRS forwarded me to this place. We would need to use libmail library to send secured mails. Thank you very much for your contribution :slight_smile:

Regards

If you visit the repo page in the first post you will see the ‘assets’ below the text - libemail.so is a directly downloadable file that can be used instead of the standard release version and will automatically enable TLS if connecting to an SMTP server port that uses it (the ‘default’ setting for SMTP_TLS).

If you are already using the standard libemail.so then you will have to put this file elsewhere and refer to it using a full path name, otherwise you can also consider overwriting it in place (but the gateway may need a restart if it ever loaded the old one).

Any issues, please post here!

Hello Peter,

Thank you for your help, I have been trying to send the mails today but unfortunately I failed on my mission, getting always the undesired RC in the log:
2022-07-26 17:48:02.794+0200 INFO: ACTION API Thread 139731673134848 return = 1 for libemail.so:SendMail(33

This is how I set the config:


<action name="Monitoring mail TLS">
    <sharedLibrary>
        <libraryFile>libemail.so</libraryFile>
        <functionName>SendMail</functionName>
        <staticParameters>
            <staticParameter>
                <name>_SMTP_SERVER</name>
                <value>
                    <data>smtp.net.intra</data>
                </value>
            </staticParameter>
            <staticParameter>
                <name>_FROM</name>
                <value>
                    <data>sender@domain.com</data>
                </value>
            </staticParameter>
            <staticParameter>
                <name>_SUBJECT</name>
                <value>
                    <data> %(_GATEWAY) - %(_MANAGED_ENTITY) %(_DATAVIEW) %(_VARIABLE) = %(_VALUE)</data>
                </value>
            </staticParameter>
            <staticParameter>
                <name>_FORMAT</name>
                <value>
                    <data>***This is an auto-generated message from ITRS Gateway %(_GATEWAY)***
-----------------------------------------------------------------
Triggered at value:  %(_VALUE)
-----------------------------------------------------------------
</data>
                </value>
            </staticParameter>
            <staticParameter>
                <name>_TO</name>
                <value>
                    <data>samuel.echevarria@domain.com</data>
                </value>
            </staticParameter>
            <staticParameter>
                <name>_SMTP_USERNAME</name>
                <value>
                    <data>sender@domain.com</data>
                </value>
            </staticParameter>
            <staticParameter>
                <name>_SMTP_PASSWORD_FILE</name>
                <value>
                    <data>mailpassword</data>
                </value>
            </staticParameter>
            <staticParameter>
                <name>_SMTP_TLS</name>
                <value>
                    <data>default</data>
                </value>
            </staticParameter>
        </staticParameters>
    </sharedLibrary>
</action>
****

If you have other configuration please let me know if I am missing a parameter.

Thank you

Regards

I do not see an SMTP_PORT in that config - for SSL/TLS you normally need to use a port other than 25 (the default). There is either the “msa” port (587) or the STARTTLS supporting port 465 (I think). Best check with your admins and/or test connectivity by hand first.

Also check the stderr output of the gateway, maybe gateway.txt if you use gatewayctl to run the gateway - it may contain more info. The challenge with writing code that makes debugging this stuff without spamming normal logs.

Also, I suggest ensuring you are calling the correct libemail.so file by using the full, absolute path in the libraryFile field.

Hello Peter,

Thank you for your feedback. May I ask you if you have an example XML of the action? Maybe I can use the same that works for you to discard other potential issues.

Thank you in advance

This one works for me (filling in the vars as appropriate):


<action name="test email">
	<sharedLibrary>
		<libraryFile>/home/peter/geneos/gateway/gateway_shared/libemail.so</libraryFile>
		<functionName>SendMail</functionName>
		<staticParameters>
			<staticParameter>
				<name>_SMTP_SERVER</name>
				<value>
					<data>smtp.gmail.com</data>
				</value>
			</staticParameter>
			<staticParameter>
				<name>_SMTP_PORT</name>
				<value>
					<data>587</data>
				</value>
			</staticParameter>
			<staticParameter>
				<name>_SMTP_USERNAME</name>
				<value>
					<var ref="username"/>
				</value>
			</staticParameter>
			<staticParameter>
				<name>_SMTP_PASSWORD_FILE</name>
				<value>
					<var ref="password_file"/>
				</value>
			</staticParameter>
			<staticParameter>
				<name>_FORMAT</name>
				<value>
					<data>&quot;Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Vivamus arcu felis bibendum ut tristique et egestas quis. Consequat nisl vel pretium lectus quam id leo in. Faucibus interdum posuere lorem ipsum dolor sit amet. Eu feugiat pretium nibh ipsum consequat nisl. Vitae tempus quam pellentesque nec. Consectetur purus ut faucibus pulvinar. Morbi tristique senectus et netus et malesuada fames. Sem viverra aliquet eget sit amet tellus cras. Pretium viverra suspendisse potenti nullam ac tortor vitae purus faucibus. Montes nascetur ridiculus mus mauris. Vulputate dignissim suspendisse in est ante in nibh mauris. Parturient montes nascetur ridiculus mus.

Fames ac turpis egestas maecenas pharetra. Aliquam sem fringilla ut morbi tincidunt augue interdum velit euismod. Libero enim sed faucibus turpis in eu. Proin sagittis nisl rhoncus mattis rhoncus urna neque viverra. Molestie at elementum eu facilisis sed odio morbi quis. Et malesuada fames ac turpis egestas integer eget aliquet nibh. Sit amet mattis vulputate enim nulla aliquet porttitor. Mi ipsum faucibus vitae aliquet nec ullamcorper. Dui faucibus in ornare quam viverra. Placerat vestibulum lectus mauris ultrices eros in cursus turpis massa. Odio euismod lacinia at quis.

Pellentesque adipiscing commodo elit at. Eget arcu dictum varius duis at. Nunc sed augue lacus viverra vitae congue eu consequat ac. Lobortis mattis aliquam faucibus purus in. Aliquet enim tortor at auctor urna nunc id cursus. Est pellentesque elit ullamcorper dignissim cras tincidunt. Lobortis feugiat vivamus at augue eget arcu dictum varius duis. Augue mauris augue neque gravida in fermentum. Massa placerat duis ultricies lacus sed turpis tincidunt id aliquet. Vitae semper quis lectus nulla at volutpat diam. Enim facilisis gravida neque convallis. Tempor orci eu lobortis elementum nibh tellus molestie.

Sed pulvinar proin gravida hendrerit lectus a. Diam vel quam elementum pulvinar etiam. Nulla posuere sollicitudin aliquam ultrices sagittis. Luctus venenatis lectus magna fringilla urna porttitor rhoncus. Vel orci porta non pulvinar neque laoreet suspendisse. In fermentum posuere urna nec. Venenatis tellus in metus vulputate. Aliquet lectus proin nibh nisl condimentum. Id consectetur purus ut faucibus pulvinar elementum integer. Ultrices sagittis orci a scelerisque purus. Purus gravida quis blandit turpis cursus. Euismod in pellentesque massa placerat duis ultricies. Justo nec ultrices dui sapien eget. Viverra mauris in aliquam sem fringilla ut morbi. Lacus suspendisse faucibus interdum posuere lorem ipsum.

Leo integer malesuada nunc vel risus. Dictum at tempor commodo ullamcorper a. Nulla facilisi cras fermentum odio eu feugiat. Consequat id porta nibh venenatis cras sed. Lectus nulla at volutpat diam ut venenatis tellus. Aliquam purus sit amet luctus venenatis lectus magna. In vitae turpis massa sed. Quis varius quam quisque id diam vel. Nulla posuere sollicitudin aliquam ultrices sagittis orci a scelerisque purus. Ullamcorper velit sed ullamcorper morbi tincidunt. Pharetra magna ac placerat vestibulum.</data>
				</value>
			</staticParameter>
			<staticParameter>
				<name>_TO</name>
				<value>
					<var ref="to"/>
				</value>
			</staticParameter>
			<staticParameter>
				<name>_FROM</name>
				<value>
					<var ref="from"/>
				</value>
			</staticParameter>
		</staticParameters>
	</sharedLibrary>
</action>

Dear Peter,
Thank you so much for your details, after several attempts I am unable to make it run :slight_smile: , do you know if the password must be specified in base64 in the file?

Thank you!! :wink:

The password should be in clear text in the password file, no encoding is supported (yet).

Worth noting that in my example XML the password file is referenced via a variable - perhaps just use an absolute path to test, e.g. /tmp/passwd.txt, and see if it any better.

With v1.2+ you can now incorporate encoded passwords (and any other credentials) in the various configuration files. This looks like this:

password: ${enc:~/.keyfile:+encs+abc1234...}

Here, the keyfile is the same as used by the Gateway for secure passwords and the +encs+... string is also the same as used by the Gateway. By default, the Gateway uses a built-in key which you cannot access. You can use the new geneos aes ... commands to create both keyfiles and encode credentials or you can use the examples in the Gateway documentation.

Because credentials need to be symmetrically encoded - i.e. they have to be decodable to send to external systems - this is not strictly “secure” but it does protect against casual viewing (‘opaquing’) and also against the config file(s) being shared without the keyfile used to encode the credentials.

Thanks to real user feedback, even if it was internal, a bug/missing feature has been addressed and there will be a v1.2.1 branch, prob a beta tomorrow, before a v1.2.x branch final release.