Archive for the 'Admin' Category

SMTP TLS Reporting, policy viewer

Screenshot of viewer webpage

For an easy way to lookup a domain’s SMTP TLS Reporting Policy try my nice MTA-STS Policy Viewer.

After writing the first notes on SMTP TLS Reporting I thought it would be nice to see all setting on one page, without using multiple tools and lookups for DNS and HTTPS data. The first iteration was a shell script, and the second iteration was a Python function. With some more fiddling I set it up as a Google Cloud Function, with a simple web frontend.

SMTP TLS Reporting, first data

The SMTP standard is just as ubiquitous as it is ossified and hard to change. Thus all newer RFCs follow similar patterns of a) adding optional extensions and b) providing feedback loops to detect usage, problems, and abuse of these extensions.

One more recent extension tries to enforce TLS transport between mailservers. This sounds very simple (and would be very simple if one could change the SMTP standard to require TLS), but is not trivial when compatibility is necessary and all TLS policies are “opt-in”.

SMTP TLS Report Chart
Read the rest of this entry »

Zabbix Data Collection Modes

I still like Zabbix as a simple allround monitoring solution. With its agent and UserParameter configuration it is very flexible and can be used (and abused) in many interesting ways. Here I want to show and compare three different patterns of collecting metrics from a service:

  1. simple item fetch
  2. fetch and send
  3. preprocessing

Read the rest of this entry »

Directory Scoped Git Configuration

I have always had a problem with the configuration of different email addresses in different git repositories. The git configuration is split into a global (~/.gitconfig) and a local/repository part (project_dir/.git/config). This works only for simple setups with one default configuration and 1-2 exceptions with per-repo overrides.

But it does not work for me, I have multiple different repositories checked out and I always forget to configure the right email address after the git clone. And it is not only “normal/private” vs. “work” address but I also have multiple addresses for different customers and open source projects. It’s a mess.

Today I finally learned about a working solution for this with git-config conditional includes. These were introduced in 2017 with git 2.13 and I found them via StackOverflow.

Read the rest of this entry »

Links 2019-01-23

On Java …

Ansible and Vagrant SSH Keys

I recently came across the question how to handle SSH keys with Vagrant and Ansible.

Vagrant

Traditionally all Vagrant boxes and VMs require a fixed login (vagrant) with a fixed SSH key. This was very convenient in a development context, but could raise security issues in case Vagrant VMs were used for anything important and users were not aware of the insecure key.

In current Vagrant versions only boxes (i. e. base images) use the insecure key. When Vagrant starts a new VM it generates a new individual SSH key for this instance. Vagrant keeps these custom keys in the .vagrant/machines subdirectory; so host to guest logins (like vagrant ssh) are still possible.

A vagrant up shows the multi-step procedure:

    workstation: Vagrant insecure key detected. Vagrant will automatically replace
    workstation: this with a newly generated keypair for better security.
    workstation: 
    workstation: Inserting generated public key within guest...
    workstation: Removing insecure key from the guest if it's present...
    workstation: Key inserted! Disconnecting and reconnecting using new SSH key...
==> workstation: Machine booted and ready!

To prevent this mechanism one can configure the VM with config.ssh.insert_key = False. This is necessary when modifying a box. Say you want to take a bento base box, install your development tools, and then repackage the VM as your own development box in order to distribute it to a development team. — With the default behaviour (config.ssh.insert_key = True) every repackage build would generate a new basebox with an individual ssh keypair; this makes the new boxes practically unusable, because every user would have to get and configure the custom SSH key for each box. With config.ssh.insert_key = False the box will retain the previous key; that means you users have the same experience as with a normal box.

Note: As a compromise between the two modes one can set a custom ssh key for all baseboxes with config.ssh.private_key_path. This might be useful for companies with many internal boxes. In this case one distribute one SSH key to use with all Vagrant boxes but not use the publicly known insecure standard key.

Ansible

If all VMs use the same SSH key the setup is straightforward: Configure all Vagrant VMs with config.ssh.insert_key = False or config.ssh.private_key_path and use that key for the Ansible login.

With Vagrant’s default behaviour there is no common SSH key for all VMs. In this case one has to configure Ansible to use the right keys for every VM, but the setup is still simple as long as you have the default /vagrant mount with the .vagrant subdirectory.

This subdirectory contains all Vagrant state and contains these files, right now the private_key is the important one:

$ find .vagrant/
.vagrant/
.vagrant/rgloader
.vagrant/rgloader/loader.rb
.vagrant/machines
.vagrant/machines/workstation
.vagrant/machines/workstation/virtualbox
.vagrant/machines/workstation/virtualbox/synced_folders
.vagrant/machines/workstation/virtualbox/vagrant_cwd
.vagrant/machines/workstation/virtualbox/index_uuid
.vagrant/machines/workstation/virtualbox/action_set_name
.vagrant/machines/workstation/virtualbox/private_key
.vagrant/machines/workstation/virtualbox/id
.vagrant/machines/workstation/virtualbox/box_meta
.vagrant/machines/workstation/virtualbox/action_provision
.vagrant/machines/workstation/virtualbox/creator_uid

I usually run one Vagrant VM as an Ansible controller (with node.vm.provision :ansible_local) and then 1 to N other VMs as installation targets. To enable access from the controller to the other VMs I include this line in the Ansible inventory: ansible_ssh_private_key_file=/vagrant/.vagrant/machines/{{ inventory_hostname }}/virtualbox/private_key

This should work just the same for running Ansible on your host machine and use it to provision the guest VMs. In this case the path would be a relative one: .vagrant/machines/....

The only important detail is the consistent naming of the VM in Vagrant and Ansible. With the ansible_ssh_private_key_file as above Vagrant’s VM name (set with config.vm.define) and Ansible’s inventory_hostname have to be the same. They are not required to match the guest’s hostname (config.vm.hostname), but I strongly recommend to either use the same value or use an obvious mapping. I usually try to use an FQDN for the hostname, and then use the short hostname (the first component) as the VM name and inventory name.

The only important detail is a consistent naming between Vagrant’s VM name (as set with config.vm.define), the VM’s hostname

postwhite

I recently updated my small mailserver and finally configured DKIM. But another change was easier and still had more impact: installing postwhite. This little tool takes a list of mail domains, then uses their SPF records to derive a list of their outgoing mail servers, then writes this list into a postscreen whitelist configuration. The current default setting contains 43 domains and generates a whitelist with nearly 2000 lines (each containing an IP or subnet). Everything is nicely scripted and can run as a nightly cronjob.

This setup eliminates my biggest problem with greylisting, which is Office356. Their combination of long email resubmit intervals and using multiple cluster servers for delivery attemps always lead to long delays before I received email from Microsoft or any company using Office356. (BTW, I really like greylisting but this is its biggest design problem: it works for single SMTP servers and enforces certain behaviour, but does not and can not consider clusters.)

Links 2017-05-15

On tools …

  • Setting the Record Straight: containers vs. Zones vs. Jails vs. VMs
    Solaris Zones, BSD Jails, and VMs are first class concepts. [..] Containers on the other hand are not real things.
  • CPU Utilization is Wrong
    The metric we all use for CPU utilization is deeply misleading, and getting worse every year. What is CPU utilization? How busy your processors are? No, that’s not what it measures. Yes, I’m talking about the “%CPU” metric used everywhere, by everyone.
  • Practical jq
    I really love jq, the JSON processor. It has changed my life and pretty much replaced Perl and Ruby as my ETL and data-munging go-to tools.
  • Reshaping JSON with jq
    Working with data from an art museum API and from the Twitter API, this lesson teaches how to use the command-line utility jq to filter and parse complex JSON files into flat CSV files.
  • A Visual Guide to What’s New in Swagger 3.0
    Over the past few years, Swagger 2 has become the de facto standard for defining or documenting your API. Since then, it’s been moved to the Linux foundation and renamed to OpenAPI Spec.
  • A plan for open source software maintainers
    As I envision it, a solution would look something like a cross between Patreon and Bugzilla: Users would be able sign up to “support” projects of their choosing […] and would be able to open issues.