Implement column projection

[gabeiglio]

This is a fix for issue #1401. In which table scans needed to infer partition column by following the column projection rules

Fixes #1401

[kevinjqliu]

Added a few comments, please take a look! The PR looks great already. Thanks for working on this!

[Fokko]

Since we only need the IDs, I'd rather just pass in those for clarity:

Suggested change

	file_project_schema: Schema,
	file_project_schema: Set[int],

[Fokko]

This is my mistake, the lookup should never be done by name, but by field-id. I've added the lookup by name to the record, but this should not be used outside of tests.

Instead, we want to create a lookup table from the field:

iceberg-python/pyiceberg/schema.py

Lines 1224 to 1233 in e646500

	def build_position_accessors(schema_or_type: Union[Schema, IcebergType]) -> Dict[int, Accessor]:
	"""Generate an index of field IDs to schema position accessors.
	Args:
	schema_or_type (Union[Schema, IcebergType]): A schema or type to index.
	Returns:
	Dict[int, Accessor]: An index of field IDs to accessors.
	"""
	return visit(schema_or_type, _BuildPositionAccessors())

It should be used something like:

accessors = build_position_accessors(table_schema)
projected_missing_fields[partition_field.name] = record[accessors[partition_field.field_id]]

[gabeiglio]

Thanks for the input! thats right, the partition field shouldn't be accessed with the name. But, if I'm understanding correctly, the partition record position in the manifest file is not the same as the position of the field in the schema.

For example, lets say I have this schema

schema {
  1 field1: string
  2 partition_field: int
}

And a DataFile like this:

DataFile {
  ...
  partition: Record(partittion_field: 1)
  ...
}

Running build_position_accessor with the schema defined earlier will result in:

{
 1: Accessor(position=0,inner=None),
 2: Accessor(position=1,inner=None),
}

Then, doing accessor[partition_field.id] will result in Accessor(position=1,inner=None) but the partition record in the manifest would be in position 0.