Using strace to Diagnose Application Problems on Linux

ps, top, and htop () tell you a process is running and how much resource it’s consuming, but not what it’s actually doing at a system call level when…

ps, top, and htop () tell you a process is running and how much resource it’s consuming, but not what it’s actually doing at a system call level when it’s misbehaving in ways that don’t show up as obvious resource spikes. strace Linux debugging goes a layer deeper, showing exactly which system calls a process makes — file access attempts, network connections, permission checks — often revealing precisely why something is failing.

What Is strace?

strace traces the system calls a running (or newly started) process makes to the Linux kernel, showing each call along with its arguments and return value. Since nearly everything a program does — opening files, reading configuration, making network connections — ultimately goes through system calls, strace provides visibility into exactly what an application is attempting, even when its own error messages are vague or unhelpful.

Installing strace

sudo apt install strace -y

Tracing a Newly Started Command

strace ./myapp

This runs myapp under strace’s supervision, printing every system call it makes as it executes, useful for understanding exactly what a misbehaving application does from the moment it starts.

Attaching to an Already-Running Process

For a process that’s already running and behaving unexpectedly:

sudo strace -p 1234

Replace 1234 with the actual PID .

A Practical Example: Diagnosing a “File Not Found” Error

When an application reports a vague error about a missing file without specifying exactly which one, strace often reveals it immediately:

strace -e trace=open,openat ./myapp

This filters output to just file-opening system calls, quickly surfacing the exact path the application attempted to open right before failing — often revealing a typo in a configuration path, or a file expected in a location that doesn’t actually exist.

A Practical Example: Diagnosing Permission Issues

Combined with the file permissions concepts , strace can pinpoint exactly which file or directory access is being denied:

strace -e trace=open,openat,access ./myapp 2>&1 | grep -i "permission denied"

This immediately identifies which specific file the application couldn’t access, rather than needing to guess based on a generic application-level error message.

A Practical Example: Diagnosing Network Connection Issues

Filter specifically for network-related system calls:

strace -e trace=network ./myapp

This reveals exactly what addresses and ports an application attempts to connect to, useful for confirming whether an application is even attempting the connection you’d expect, or failing before it gets that far.

Understanding Common strace Output Patterns

  • ENOENT – “No such file or directory,” indicating the application tried to access a path that doesn’t exist
  • EACCES – “Permission denied,” pointing directly to a file permissions issue
  • ECONNREFUSED – a network connection attempt was actively rejected, often indicating the target service isn’t actually listening on the expected port
  • ETIMEDOUT – a connection attempt timed out, potentially indicating a firewall rule (UFW, ) silently dropping the traffic rather than actively rejecting it

Measuring Time Spent in System Calls

For performance troubleshooting rather than error diagnosis, strace can show timing information:

strace -T ./myapp

This appends the time spent in each system call, useful for identifying whether an application is spending unexpected time waiting on a specific file operation or network call, which pure resource monitoring (CPU/memory usage) wouldn’t directly reveal.

Tracing Only Specific Syscall Categories

For a less overwhelming output on complex applications making many system calls, filter to specific categories relevant to your investigation:

strace -e trace=file ./myapp
strace -e trace=network ./myapp
strace -e trace=process ./myapp

When to Reach for strace vs Other Tools

  • Resource usage seems fine but the application still isn’t working correctly → strace, to see exactly what it’s attempting
  • Vague “permission denied” or “file not found” errors → strace, to identify the specific exact path involved
  • General resource consumption questionshtop/top remain the better starting point
  • Understanding application logs and behavior over time → Loki or journalctl for historical context strace’s live tracing doesn’t provide

Final Thoughts

strace Linux debugging provides real system-call-level visibility that resource monitoring and application logs alone often can’t surface, particularly for vague permission, missing-file, or network connection errors that don’t show up as obvious resource anomalies. Combined with the process management and log analysis tools covered elsewhere in this series, strace fills a genuinely useful diagnostic gap for the specific class of problems where “something is failing but the error message isn’t specific enough” describes the situation exactly.

Related Posts

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *