From Page To Snapshot

The write path is where Trino’s Page model turns into Iceberg files and Iceberg metadata.

For CTAS (Create Table As Select) and INSERT, the compact mental model is:

workers write data files
coordinator commits Iceberg metadata

That split matters. A write is not just “Trino creates a Parquet file.” Workers receive page batches, write data files through the connector page sink, and return small commit fragments. The coordinator gathers those fragments and asks Iceberg to commit a new snapshot.

The shape I want to remember is:

upstream operators
  -> Page
  -> TableWriterOperator
  -> IcebergPageSink.appendPage(...)
  -> IcebergPageSink.finish()
  -> commit fragments
  -> TableFinishOperator
  -> IcebergMetadata.finishCreateTable(...) or finishInsert(...)
  -> Iceberg snapshot and manifest metadata

This note only covers the append-style write path: CTAS and INSERT.

1. The Setup

Use a small Iceberg table so the write is easy to inspect:

DROP TABLE IF EXISTS iceberg.write_trace.orders_delta;
DROP SCHEMA IF EXISTS iceberg.write_trace;

CREATE SCHEMA iceberg.write_trace
WITH (location = 'local:///write-trace');

Then create the table with CTAS:

CREATE TABLE iceberg.write_trace.orders_delta
WITH (
    format = 'PARQUET',
    format_version = 2,
    partitioning = ARRAY['orderstatus']
) AS
SELECT
    orderkey,
    custkey,
    orderstatus,
    totalprice
FROM (
    VALUES
        (1, 1001, 'O', CAST(10.00 AS DOUBLE)),
        (2, 1002, 'P', CAST(20.00 AS DOUBLE)),
        (3, 1003, 'F', CAST(30.00 AS DOUBLE)),
        (4, 1004, 'O', CAST(40.00 AS DOUBLE))
) AS t(orderkey, custkey, orderstatus, totalprice);

This statement is useful because it is both:

CREATE TABLE:
  define the Iceberg table schema, partitioning, and location

AS SELECT:
  run a query that produces rows and writes them into the new table

So CTAS is DDL plus a write query. It needs a table-create path and a data-write path.

2. The Plan Shape

The useful distributed EXPLAIN shape for CTAS is roughly:

Fragment 0 [COORDINATOR_ONLY]
  TableCommit[target = iceberg:write_trace.orders_delta@...]
    RemoteSource[sourceFragmentIds = [1]]

Fragment 1 [SOURCE]
  TableWriter[target = iceberg:write_trace.orders_delta@...]
    Values[]
      (1, 1001, 'O', 10.0)
      (2, 1002, 'P', 20.0)
      (3, 1003, 'F', 30.0)
      (4, 1004, 'O', 40.0)

The exact symbol names, fragment labels, and writer distribution can vary. The important operators are:

Read the data flow from the source fragment toward fragment 0:

Fragment 1
  Values produces pages
  TableWriter writes data files and emits fragments

Fragment 0
  TableCommit consumes fragments
  connector commits the Iceberg snapshot

That is the same fragment-reading habit as the EXPLAIN post: follow RemoteSource, not printed order.

3. Phase 1: Workers Write Data Files

The worker-side write boundary is:

upstream operator output
  -> Page
  -> TableWriterOperator.addInput(Page)
  -> ConnectorPageSink.appendPage(Page)

For this CTAS, the upstream operator is simple:

ValuesOperator
  -> TableWriterOperator

For a CTAS from another table, the upstream side could be a scan:

TableScanOperator
  -> filter/project work
  -> TableWriterOperator

For a more complex INSERT SELECT, the upstream side could include joins, aggregations, exchanges, or sorts. The writer does not care where the page came from. It receives a Page batch and passes the rows to the connector sink.

The connector handoff is:

TableWriterOperator
  -> PageSinkManager
  -> IcebergPageSinkProvider
  -> IcebergPageSink

At this point, Trino is crossing from engine execution into the Iceberg connector’s writer.

4. What IcebergPageSink Does

IcebergPageSink.appendPage(...) receives Trino pages and writes rows into Iceberg data files.

A Page is not a file. It is an in-memory batch of rows in Trino’s columnar Block format.

The page sink has to turn those pages into file output:

incoming Page
  -> choose writer or partition writer
  -> write rows into Parquet
  -> close file writers at finish
  -> return CommitTaskData fragments

For the CTAS table:

partitioning = ARRAY['orderstatus']

the sink may write rows into different partition outputs. The physical file names and counts are not the lesson. The lesson is that worker tasks write data files before the coordinator commits the table metadata.

When worker-side writing finishes:

IcebergPageSink.finish()
  -> closes file writers
  -> returns serialized commit fragments

Those fragments are small receipts. They describe written files. They are not the table snapshot by themselves.

5. Phase 2: Coordinator Commits Metadata

The coordinator-side boundary is:

TableFinishOperator
  -> metadata finish method
  -> Iceberg commit

In the plan, this is the TableCommit fragment:

Fragment 0 [COORDINATOR_ONLY]
  TableCommit
    RemoteSource[sourceFragmentIds = [1]]

The coordinator receives writer fragments from workers, gathers them, and calls the connector finish path.

For CTAS, the finish route is:

TableFinishOperator
  -> finishCreateTable(...)
  -> IcebergMetadata.finishCreateTable(...)

For CTAS with rows, the create-table finish path still has to commit the data files. The Iceberg side turns worker fragments into DataFile records and commits an append-style snapshot.

Conceptually:

commit fragments
  -> DataFile objects
  -> AppendFiles
  -> new Iceberg snapshot
  -> new manifest metadata

This is the part that makes Iceberg different from “write some files under a directory.” The table state is the committed snapshot. The data files become visible because the snapshot metadata points to them.

6. INSERT Reuses The Same Shape

After CTAS, append more rows:

INSERT INTO iceberg.write_trace.orders_delta
VALUES
    (5, 1005, 'F', CAST(50.00 AS DOUBLE)),
    (6, 1006, 'O', CAST(60.00 AS DOUBLE));

INSERT is simpler than CTAS because the table already exists.

The begin path changes:

CTAS:
  beginCreateTable(...)
  create a writable table handle for a new table

INSERT:
  beginInsert(...)
  create a writable table handle for an existing table

But the worker write model is the same:

upstream pages
  -> TableWriterOperator
  -> IcebergPageSink.appendPage(...)
  -> IcebergPageSink.finish()
  -> commit fragments

The coordinator finish route changes:

CTAS:
  finishCreateTable(...)

INSERT:
  finishInsert(...)

The Iceberg commit concept is still append-style:

new data files
  -> AppendFiles
  -> new snapshot

So INSERT is not a special row-by-row mutation. It writes new files and commits new metadata that makes those files part of the current table snapshot.

7. What To Check In Iceberg

After CTAS, check that the table exists:

SHOW CREATE TABLE iceberg.write_trace.orders_delta;

Check snapshots:

SELECT
    committed_at,
    snapshot_id,
    parent_id,
    operation,
    summary
FROM iceberg.write_trace."orders_delta$snapshots"
ORDER BY committed_at;

After CTAS, one snapshot should be there for the initial append. After INSERT, there should be another snapshot.

Check current files:

SELECT
    content,
    file_path,
    file_format,
    record_count,
    partition
FROM iceberg.write_trace."orders_delta$files"
ORDER BY content, file_path;

For this narrow post, the important check is:

CTAS:
  table exists
  snapshot exists
  data files exist

INSERT:
  another snapshot exists
  additional data-file work is visible

8. What The Evidence Proves

The evidence types prove different things:

For a read query, EXPLAIN ANALYZE reads data and reports runtime stats. For a CTAS or INSERT, it runs the write. Use it only when writing to a disposable trace table.

9. Code Anchors

These are the source areas to inspect after the plan shape makes sense:

The useful first debugger path is:

BeginTableWrite
  -> IcebergMetadata.beginCreateTable(...) or beginInsert(...)
  -> TableWriterOperator.addInput(Page)
  -> IcebergPageSink.appendPage(Page)
  -> IcebergPageSink.finish()
  -> TableFinishOperator
  -> IcebergMetadata.finishCreateTable(...) or finishInsert(...)

That path is enough to understand CTAS and INSERT. It is not enough for DELETE or MERGE, because row-level changes use a merge sink and RowDelta commit.

10. CTAS vs INSERT

The shared part is larger than the different part:

both use TableWriterOperator
both use IcebergPageSink
both return commit fragments
both finish on the coordinator
both make data visible through Iceberg snapshot metadata

11. What To Remember

• CTAS is create-table plus write-data.
• INSERT reuses the same append-style write model against an existing table.
• Workers write physical data files through IcebergPageSink.
• Workers return commit fragments, not final table metadata.
• The coordinator commits those fragments into Iceberg snapshot and manifest metadata.
• The durable Iceberg table state is the snapshot, not just the files sitting in storage.
• DELETE and MERGE are harder because they are row-change writes. CTAS and INSERT are the simpler foundation.