Iceberg functions doesnt work

[kosmldev]

Query engine

Spark 3.5, Hive Metastore

Question

Functions CALL doesnt work, also doesnt work creating TAG and branches

my code:

conf = SparkConf()
conf.set('spark.hadoop.fs.s3a.access.key', s3_secret['aws_access_key_id'])
conf.set('spark.hadoop.fs.s3a.secret.key', s3_secret['aws_secret_access_key'])
conf.set('spark.hadoop.fs.s3a.endpoint', s3_secret['endpoint_url'])
# Hive Metastore
conf.set(f'spark.hadoop.{hms_bucket}.access.key', hms_secret['aws_access_key_id'])
conf.set(f'spark.hadoop.{hms_bucket}.secret.key', hms_secret['aws_secret_access_key'])

conf.set('spark.sql.extensions','org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions')
conf.set("spark.sql.catalog.metastore_db", "org.apache.iceberg.spark.SparkCatalog")
conf.set("spark.sql.iceberg.check-ordering", "false") 
conf.set("spark.sql.catalog.metastore_db.type", "hive")
conf.set("spark.sql.catalog.metastore_db.uri", elf.hms_secret['hive_uris'])

conf.set("spark.jars.packages", "org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.6.1,org.postgresql:postgresql:42.7.2")

spark = (
    SparkSession.builder
    .config(conf=conf)
    .appName('iceberg')
    .getOrCreate()
)

# table created like
test.spark.sql("""
    CREATE TABLE metastore_db.spark_iceberg.test_funcions (
        store_id int, item_id int, dt date, smth float
    )
    USING iceberg
    PARTITIONED BY (
        bucket(5, store_id),
        bucket(5, item_id),
        year(dt)
    );
""")

# snapshots
test.spark.sql('select * from metastore_db.spark_iceberg.test_funcions.snapshots').show()

# +--------------------+-------------------+-------------------+---------+--------------------+--------------------+
# |        committed_at|        snapshot_id|          parent_id|operation|       manifest_list|             summary|
# +--------------------+-------------------+-------------------+---------+--------------------+--------------------+
# |2025-01-13 03:49:...|3581078148379069374|               null|   append|s3a://t-hive-meta...|{spark.app.id -> ...|
# |2025-01-13 03:49:...|1530700004686346841|3581078148379069374|overwrite|s3a://t-hive-meta...|{spark.app.id -> ...|
# |2025-01-13 03:49:...|4471429915878262482|1530700004686346841|overwrite|s3a://t-hive-meta...|{spark.app.id -> ...|
# |2025-01-13 03:49:...|5729427534040942866|4471429915878262482|overwrite|s3a://t-hive-meta...|{spark.app.id -> ...|
# +--------------------+-------------------+-------------------+---------+--------------------+--------------------+

# None of this doesn't work
# https://iceberg.apache.org/docs/latest/spark-procedures/
# https://iceberg.apache.org/docs/latest/spark-ddl/
# test.spark.sql("CALL metastore_db.system.rollback_to_snapshot('spark_iceberg.test_funcions', 1530700004686346841);")
# test.spark.sql("CALL metastore_db.system.expire_snapshots('spark_iceberg.test_funcions', 1530700004686346841);")
# test.spark.sql("CALL metastore_db.system.rewrite_data_files('spark_iceberg.test_funcions');")
# test.spark.sql(f"ALTER TABLE metastore_db.spark_iceberg.test_funcions CREATE TAG 'random tag' AS SNAPSHOT 1530700004686346841;")

Maybe i have wrong session config?

[nastra]

What error are you getting when you e.g. call the expire_snapshots procedure or when you create a tag?