Logging & Debugging

Debugging in Python

In order to debug an error in Python, we can use debuggers and, in addition, add logging to the application.

Logging

Syslog Integration

• Concept: Writing basic debug messages directly to the Linux operating system’s internal logging daemon.
• Methods & Syntax:
• syslog(message): Sends the string to the system logger.

Program:

debug.py

from syslog import syslog

syslog('This is a debug message.')

Execution & Verification (Linux Terminal):

$ python debug.py
$ tail /var/log/syslog

---

The Logging Module & Severity Levels

• Concept: Python’s standard logging module categorizes messages by severity. It is the industry standard for tracking application flow and state. By default, the root logger only outputs messages at the WARNING level or higher.
• Methods & Syntax:
  • logging.basicConfig(level=logging.LEVEL): Modifies the minimum threshold for what gets processed.
  • logging.debug('msg'): Detailed information, typically of interest only when diagnosing problems.
  • logging.info('msg'): Confirmation that things are working as expected.
  • logging.warning('msg'): An indication that something unexpected happened, but the software is still working. (Default minimum level).
  • logging.error('msg'): Due to a more serious problem, the software has not been able to perform some function.
  • logging.critical('msg'): A serious error, indicating that the program itself may be unable to continue running.

Program:

import logging

# Set the threshold to INFO (skips DEBUG)
logging.basicConfig(level=logging.INFO)

logging.info('This will get logged.')
logging.debug('This will NOT get logged.')
logging.critical('System failure imminent.')

# Output:
# INFO:root:This will get logged.
# CRITICAL:root:System failure imminent.

---

File Output and Formatting

• Concept: Redirecting log outputs from the console to a dedicated file and customizing the exact structure of the log string using built-in variables.
• Methods & Syntax:
  • filename='app.log': The target file.
  • filemode='w': How to open the file ('w' overwrites, 'a' appends).
  • format='...': The string structure. Standard variables include %(name)s (logger name), %(levelname)s (severity), and %(message)s (the actual log text).

Program:

import logging

logging.basicConfig(
    filename='app.log',
    filemode='w',
    format='%(name)s - %(levelname)s - %(message)s'
)

logging.error('Database connection timed out.')

# Result inside app.log:
# root - ERROR - Database connection timed out.

---

Complex Configuration via External Files

• Concept: Hardcoding log settings is inefficient for large applications. Best practice involves defining formatters, handlers (where logs go, like files or consoles), and specific loggers in an external configuration file (like YAML) and loading it dynamically.

• Methods & Syntax:

  • yaml.safe_load(file): Parses the YAML file into a Python dictionary.
  • logging.config.dictConfig(config): Applies the dictionary configuration to the logging system.
  • logging.getLogger("name"): Instantiates and retrieves a specific logger defined in the config file.

  Program:

  1. The Configuration File (config.yaml)

  version: 1
  formatters:
    simple:
      format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
  handlers:
    console:
      class: logging.StreamHandler
      level: DEBUG
      formatter: simple
      stream: ext://sys.stdout
  loggers:
    simpleExample:
      level: DEBUG
      handlers: [console]
      propagate: no
  root:
    level: DEBUG
    handlers: [console]

  2. The Python Script

  import logging
  import logging.config
  import yaml
  
  # Load the YAML configuration
  with open('config.yaml', 'r') as f:
      config = yaml.safe_load(f.read())
      logging.config.dictConfig(config)
  
  # Call the specific logger defined in the YAML file
  logger = logging.getLogger("simpleExample")
  
  logger.debug('This is a debug message configured via YAML.')
  
  # Output:
  # 2026-03-14 16:08:10,000 - simpleExample - DEBUG - This is a debug message configured via YAML.