How to View DNS Calls Made by Processes

· by Victor Mendonça · Read in about 3 min · (516 words) ·

We had the need at work to monitor DNS calls made by an application in a RHEL system in order to stabilish if a connection pool config change had taken full effect, or if we had missed any configuration file. And the solution was to use SystemTap for this task.

SystemTap (stap) is a scripting language and tool that simplifies the gathering of information about the running Linux system. It allows you to monitor and trace the operation of a Linux kernel.

It should be present on most RHEL server installs, but for other desktop based distros (like Arch), you might need to install it.

Overview

The image below shows you the sript working and recording DNS lookup from Firefox (note the difference on the amount of connections between Google/Yahoo vs DuckDuckGo).

Installation

Install systemtap and linux-headers:

pacman -Syu systemtap linux-headers

Initial Config and Test

Because we are trying to figure out how the application makes DNS calls, we need to find where our libc6 library lives so we can probe it for requests (most likelly /usr/lib/libc.so.6 for Arch).

pacman -Ql glibc | grep '/libc.*so'
glibc /usr/lib/libc-2.26.so
glibc /usr/lib/libc.so
------------------------
glibc /usr/lib/libc.so.6
------------------------
glibc /usr/lib/libcidn-2.26.so
glibc /usr/lib/libcidn.so
glibc /usr/lib/libcidn.so.1
glibc /usr/lib/libcrypt-2.26.so
glibc /usr/lib/libcrypt.so
glibc /usr/lib/libcrypt.so.1

To test it, make sure the string probe process("/usr/lib/libc.so.6") has the location for libc6 on your system and run the command below:

sudo /usr/bin/stap -e 'probe process("/usr/lib/libc.so.6").function("getaddrinfo") { log(execname()) }'

You may get the following warning when running the script:

WARNING: Kernel function symbol table missing [man warning::symbols]

This is because Systemtap may need a linux-build style System.map file to find addresses of kernel functions/data. Try the command below to create it by hand:

sudo cp /proc/kallsyms /boot/System.map-`uname -r`

Running It


You can run the script below (or just the code) as root to monitor the connections. Output will be displayed on your current terminal, or you can choose to save it to a file.

_#!/bin/bash


system_stap="/usr/bin/stap"

_getaddrInfo () {
  /usr/bin/stap -e 'probe process("/usr/lib/libc.so.6").function("getaddrinfo")
{
  printf("| %-15s| %-7d| %-35s |\n", execname(), pid(), kernel_string(pointer_arg(1)))
}'
}

echo ""
printf "| %-15s| %-7s| %-35s |\n" "Process" "PID" "Destination Name"
echo "|----------------|--------|-------------------------------------|"


while true ; do
  _getaddrInfo
done
probe
  process("/lib64/libc.so.6").function("__gethostbyname_r").call,
  process("/lib64/libc.so.6").function("gethostbyname").call,
  process("/lib64/libc.so.6").function("__gethostbyname2_r").call,
  process("/lib64/libc.so.6").function("gethostbyname2").call,
  process("/lib64/libc.so.6").function("__new_gethostbyname2_r").call
{
	printf("[%s][%d]->%s(%s)\n", execname(), pid(), user_string(pointer_arg(1)), kernel_string(pointer_arg(1)))
}

Output:

$ sudo ./getAdreessInfo.sh
[sudo] password for victor:           

| Process        | PID    | Destination Name                    |
|----------------|--------|-------------------------------------|
| WorkerPool/5863| 5104   | adservice.google.ca                 |
| WorkerPool/6005| 5104   | apis.google.com                     |
| WorkerPool/1335| 5104   | clients5.google.com                 |
| WorkerPool/5823| 5104   | notifications.google.com            |
| WorkerPool/1856| 5104   | lh3.googleusercontent.com           |
| WorkerPool/5297| 5104   | ogs.google.com                      |
| WorkerPool/5823| 5104   | www.google.com                      |
| WorkerPool/1335| 5104   | www.gstatic.com                     |
| WorkerPool/5863| 5104   | uaswitcher.org                      |
| WorkerPool/6005| 5104   | fonts.gstatic.com                   |
| WorkerPool/5823| 5104   | play.google.com                     |

Closing Notes


The uses for SystemTap are very wide and diverse. You will be able to find great tutorials and documentation online. I’m also providing a few links below to some great documentation to get you started.

code with