# Observability

Three hooks report what the driver does: a **query logger** for every statement, **tracing** through
OpenTelemetry, and a **notice listener** for what the server says along the way. All three are
process-wide, so set them up once, when the application starts.

## Query logging

```pycon
>>> records = []
>>> agensgraph.add_query_logger(records.append)
>>> conn.execute_query("MATCH (m:movie) WHERE m.year = %s RETURN m.title", (1999,)).records
[('The Matrix',)]
>>> agensgraph.remove_query_logger(records.append)
>>> record = records[0]
>>> record.statement, record.rows, record.failed
('MATCH (m:movie) WHERE m.year = %s RETURN m.title', 1, False)
```

The logger is called with a `QueryRecord` for each statement:

| Field | |
|---|---|
| `statement` | the statement as sent, with its placeholders; parameter values are never included |
| `elapsed` | seconds from sending the statement until the server answered, not including reading the rows |
| `rows` | how many rows it returned or changed; 0 for a failure |
| `failed` | whether it raised |
| `error` | the exception, for a failure |
| `connection` | a number identifying the connection, never its settings, which can hold a password |

**Every statement is reported**, not only those `execute_query()` sends: `conn.execute()`, a cursor of
your own, `stream()`, and the driver's own statements, such as catalog reads, the statements behind a
deadline, and the counters behind `counts_=True`. Someone asking what the driver sends wants the round
trips they did not write. The data of a `COPY`, which the [bulk loaders](../data/bulk-loading.md) send,
is not a statement and is not reported.

With no logger attached, logging costs a flag test per statement.

## Tracing

With the `otel` extra installed, `enable_tracing()` takes a span for each statement through
OpenTelemetry's API:

```{code-block} python
:class: no-run

agensgraph.enable_tracing()            # the tracer named "agensgraph" from the global provider
agensgraph.enable_tracing(my_tracer)   # or one you give it
```

Each span is named `query` and carries `db.system.name` (`postgresql`), `db.query.text` (the
statement), and `db.namespace` (the graph, when the connection has selected one). Parameter values never
reach a span. Spans are taken by `execute_query()`; statements sent through `execute()` or a cursor of
your own are seen by the query logger but are not traced.

The driver imports only `opentelemetry-api`, and only when tracing is turned on; it never imports an
SDK. Configure the provider and exporter in your application as usual. `disable_tracing()` turns it off.
Without the extra, `enable_tracing()` raises `RuntimeError` saying what to install.

## Notices

PostgreSQL sends notices and warnings alongside a statement's result, for example from a function's
`RAISE NOTICE`. A notice listener receives each one as a `Notice`:

```pycon
>>> from agensgraph.observability import add_notice_listener, remove_notice_listener
>>> notices = []
>>> add_notice_listener(notices.append)
>>> conn.execute("DO $$ BEGIN RAISE NOTICE 'hello from the server'; END $$")
<agensgraph.connection.Cursor ...>
>>> remove_notice_listener(notices.append)
>>> notices
[Notice(severity='NOTICE', code='00000', message='hello from the server', detail=None, hint=None)]
```

`severity` is the non-localised severity, `code` the SQLSTATE, and `str(notice)` reads
`NOTICE: hello from the server`. A listener is told about notices from every connection the driver
makes.

## Keep callbacks short

A logger or a listener is called on whatever ran the statement, as it happens: the thread, or on the
awaiting interface the task, so a callback that blocks holds up the statement, and in awaiting code the
whole event loop. Make each one put a value somewhere and return: append to a list, increment a counter,
or put onto a queue that something else drains. Writing to a socket or a file belongs on the other side
of that queue.

A callback that raises is logged, through the `agensgraph` logger, and otherwise ignored: it cannot
replace the statement's own outcome, and one that raises does not stop the others being called.