Prometheus (NathanDimmer) Mac OS

‎Zeus got angry and punished the Titan Prometheus because he gave to the mankind the gift of fire and the skill of the metalwork. ‎Kids 2016 Global Nav Open Menu Global Nav Close Menu. I was on Mac OS and using a Docker image of Prometheus (version=1.7.1) When i went into the docker container and checked the dateTime, walaaaaaaaaaaaa There was a Time mismatch between host and Docker container. Mac OS Many computers today come with support for 'hardware virtualization' (called VT-x by Intel and AMD-V by AMD ), which improves performance of virtual machines. On Macs, it’s enabled by default, in which case there’s no need to read further!

  1. Prometheus (nathandimmer) Mac Os X
  2. Prometheus (nathandimmer) Mac Os Download
  3. Prometheus (nathandimmer) Mac Os Catalina

This is the sixth part in my ongoing series on using SaltStack to deploy Consul and Prometheus Exporters, enabling Prometheus to discover machines and services to monitor. You can view the other posts in the series below: -

All of the states are available in my Salt Lab repository.

Why MacOS?

Compared to the other systems covered so far, MacOS might seem an odd choice, as it is primarily a desktop operating system. However there are many cases where businesses have fleets of Mac devices requiring centralized management and monitoring that is scalable: -

  • A render farm for video and/or animation
  • Continuous Integration pipelines for iOS applications
  • Building and signing MacOS applications

Companies like MacStadium offer Macs to customers in a cloud-like fashion for these kind of workloads, and there are other places (like in this article) that use Macs as part of their rendering workflow.

MacOS has its roots in Unix (FreeBSD and NeXTSTEP) so in some ways it is similar managing a BSD or illumos system. There are some differences in service management, but those used to Linux or Unix should not feel entirely out of place on the MacOS command line.

Configuring MacOS

MacOS takes you through the steps of setting up your machine during the install process. If you need to set a static IP after installation, go to Applications > System Preferences > Network, and then set the IP under the correct interface: -

Installing the Salt Minion

To install the Salt Minion in MacOS, download the latest version from here, and then install by opening the executable and following the onscreen instructions. Once it is installed, you could configure it using: -

Configuring the Salt Minion

Salt has an included minion configuration file. We replace the contents with the below: -

/etc/salt/minion

You can then restart the Salt Minion using: -

MacOS’s launchctl command does have a restartFirepunk mac os. option, hence the need to stop and start the service.

You should now see this host attempt to register with the Salt Master: -

Accept the host with salt-key -a 'macos-01*'. Once this is done, you should now be able to manage the machine using Salt: -

Darwin is the base kernel that MacOS, iOS and Apple’s other OSs are built upon.

Salt States

We use two sets of states to deploy to MacOS. The first deploys Consul. The second deploys the Prometheus Node Exporter. Project fireball mac os.

Applying Salt States

Once you have configured the states detailed below, use one of the following options to deploy the changes to the MacOS machine: -

  • salt '*' state.highstate from the Salt server (to configure every machine and every state)
  • salt 'macos*' state.highstate from the Salt server (to configure all machines with a name beginning with `macos*, applying all states)
  • salt 'macos*. state.apply consul from the Salt server (to configure all machines with a name beginning with macos*, applying only the consul state)
  • salt-call state.highstate from a machine running the Salt agent (to configure just one machine with all states)
  • salt-call state.apply consul from a machine running the Salt agent (to configure just one machine with only the consul state)

You can also use the salt -C option to apply based upon grains, pillars or other types of matches. For example, to apply to all machines running an MacOS kernel, you could run salt -C '[email protected]:Darwin' state.highstate.

Consul - Deployment

The following Salt state is used to deploy Consul onto a MacOS host: -

This state is similar to the states used for Linux or illumos.

Hashicorp provide MacOS binaries for Consul, so we can use the archive.extracted state to download it and extact it. After that, the main differences from the other states are how MacOS defines and uses services.

The service is defined in a plist file, which looks like the below: -

/srv/salt/states/consul/client/files/consul-launchdaemon

Like illumos/Solaris, MacOS defines services using XML. The relevant fields are: -

  • Key: Label - This is the name of the service (com.consul.consul)
  • RunAtLoad: True - This ensures the service starts on boot
  • KeepAlive: True - This makes sure that the service keeps running, rather than being a one time execution
  • ProgramArguments - This contains the parameters we use to run Consul (with each argument on a different line)

There are also other paramters like Soft and Hard Resource limits (which control how many files can be open on the system, and how many can be open per process), as well as some debugging options.

The command launchctl load /Library/LaunchDaemons/com.consul.consul.plist imports the plist file. This is only reimported if the contents of the service file change.

After that, the service is stopped and started, to ensure it will pick up any changes made to the service in future.

The final section regarding Consul services is the same as we use in the other parts in this series. This adds all services that Prometheus can then discover.

This state is applied to MacOS machines as such: -

We match the kernel grain, ensuring the value is Darwin: -

Pillars

We use the same pillars for MacOS as we do for Linux, OpenBSD and illumos: -

consul.sls

consul-dc.sls

These pillars reside in /srv/salt/pillars/consul. They are applied as such: -

To match MacOS, we add the [email protected]:Darwin part to our original match statement (to include the standard consul.consul pillar).

Consul - Verification

We can verify that Consul is working with the below: -

Node Exporter - Deployment

Now that Consul is up and running, we will install the Prometheus Node Exporter.

Prometheus

States

The following Salt state is used to deploy the Prometheus Node Exporter onto a MacOS host: -

This state is very similar to the Linux state.

The first part downloads the node_exporter binary from Github, and extracts the contents. This extracts the contents to /usr/local/bin/node_exporter-1.0.0.darwin-amd64/. We then move the node_exporter binary into /usr/local/bin directory.

Finally, we clean up the the directory created by the archive, and remove the archive as well. These steps are executed if the node_exporter binary is not already /usr/local/bin.

After that, we create the Node Exporter user, group, textfile directory, and then define the service (using plist files like in the Consul deployment).

The contents of the plist are: -

/srv/salt/states/exporters/node_exporter/files/node_exporter-launchdaemon

This is very similar to the Consul service. The main differennces are: -

  • The label is com.prometheus.node_exporter (used when restarting the service)
  • The ProgramArguments (as it is a different service)

We apply the state with the following: -

Pillars

There are no pillars in this lab specific to the Node Exporter.

Node Exporter - Verification

After this, we should be able to see the node_exporter running and producing metrics: -

All looks good!

Prometheus Targets

As Prometheus is already set up (see here), and matches on the prometheus tag, we should see this within the Prometheus targets straight away: -

The last host in the list is the MacOS machine

Above is the Metadata we receive from Consul about this host.

Grafana

If we use the Node Exporter Full dashboard, most of the metrics are available: -

In the above, the graphs for memory are missing. This is where we can use some of the BSD heritage of MacOS, by using the Node Exporter FreeBSD dashboard: -

While we are still missing some metrics, we have more available than in the previous dashboard. With some tweaking to this dashboard, we can see all of the values: -

This does mean that the dashboard would no longer work for FreeBSDs hosts, so you would need to duplicate it and update the graphs accordingly.

Summary

If, like me, you have used Macs for a number of years, you have probably used GUI utilities to manage them. However the capabilities are available to managem them like any other server operating system. Using something like SaltStack allows you to integrate MacOS alongside other systems with little effort.

Prometheus (nathandimmer) Mac Os X

As with all the systems we have covered, using the same tools to configure, manage and monitor your infrastructure is invaluable. If your infrastructure includes the need to build iOS applications, or you are managing a render farm in conjunction with Linux, Windows, BSD or illumos, you can use Prometheus and Grafana as your one pane of glass to monitor everything.

This post is the last in the series on managing and monitoring different operating systems using SaltStack, Consul and Prometheus. I hope those who read this series find it useful, and I am more than willing to include systems not already covered in future posts (suggestions in the comments!).

hi there,

recently i started pushing metrics to prometheus but whenever i sent a sample query to prometheus to see the data points, i would get none.

There were no errors in prometheus and i was scratching my head …. grrr…

I was on Mac OS and using a Docker image of Prometheus (version=1.7.1)

When i went into the docker container and checked the dateTime, walaaaaaaaaaaaaLavender link mac os.

There was a Time mismatch between host and Docker container.

Mac

Prometheus (nathandimmer) Mac Os Download

#___7_day__difference__ –> Possible docker bug !!!

since i was searching in the current timebucket, and the datapoints were going to time bucket 7 days ago, i was not able to find em.

FIX:

Prometheus (nathandimmer) Mac Os Catalina

Restarting docker for Mac fixed the issue.