Skip to content

Amoro-0.6.1
Compare
Choose a tag to compare
@zhoujinsong zhoujinsong released this 21 Feb 07:39
· 569 commits to master since this release
v0.6.1
0675723

Highlight:

  • Improve the performance of the optimizer in cases where there are a large number of delete files

Improvements:

  • Improve optimizer's compaction performance in large delete scenarios #2266
  • Store max transaction id on the snapshot level for Mixed Format Table #1720
  • Reduce triggering of optimizating evaluation for native iceberg format #2350
  • The mixed hive tables should be allowed to be dropped when the hive table has already been dropped #2286
  • Print GC date stamps #2416
  • Exclude kryo dependency from flink-optimizer #2437

BugFixes:

  • Fix data lost issue after optimizing for Mixed format tables #2253
  • After disabling self-optimizing, the table status is not updated to idle #2250
  • Optimizing status blocked at minor/major when restarting AMS #2279
  • Exception happened when using Flink DataStream API to write data into Mixed format tables #2271
  • Failure when creating a table and immediately retrieving Blocker #2289
  • Fix deleting Puffin files by mistake when cleaning orphan files #2320
  • There is a not daemon thread exists when the Spark job finished #2336
  • SimpleDateFormat throws parsing errors when using multi-threading #2324
  • Dashboard displays wrong partition path when the table had multiple partition spec #1863
  • Fix the NullPointerException issue when modifying the self-optimizing.enabled configuration #2388
  • TableController set partition string value to null when getting file list for a non-partitioned table #2407
  • Fix loading the optimizing snapshot id of ChangeStore for Mixed Format tables#2430
  • Batch deletion of ChangeStore files did not take effect for the Mixed Format tables #2440
  • Fix File format is required when scanning file entries if no file format is specified in the file name #2441
  • Mixed Hive tables mistakenly delete hive files during expiring snapshots #2404
  • Using the wrong database name when switching to another Catalog #2413
  • Fix Mixed Format tables expiring all the snapshots with optimized sequence #2394
  • Add a default value of table-filters in spark scan builder to avoid NPE when comparing the table scan #2313
  • Failed to get table properties when BaseStore contains property watermark.base #2295
  • Show create table statement return incorrect SQL with Spark #2272
  • AMS will repeat trigger the same minor optimizing jobs when the bucket > self-optimizing.minor.trigger.file-count #2464
  • Periodic snapshots expiring tasks will be scheduled redundantly #2453
Assets 11
Loading