Add support for decimal256 and allow rounding decimal to scale

[raghumdani]

Summary

Arrow cannot support reading CSV using decimal256. We add support by first reading them as strings and then casting to decimal256. Secondly, we support rounding decimal values to scale specified in the schema if the decimal values in the CSV does not conform to scale.

Rationale

The behavior will be same as Spark compaction.

Changes

• added tests
• Changed pyarrow.py

Impact

This change is backward compatible.

Testing

Changes were tested in a functional test suite as well reading individual problematic files in beta.

Regression Risk

Low

Checklist

• Unit tests covering the changes have been added

  • If this is a bugfix, regression tests have been added

• E2E testing has been performed

Additional Notes

Any additional information or context relevant to this PR.

[yankevn]

Could you also test on a table that was failing previously?

[raghumdani]

Could you also test on a table that was failing previously?

Yes, tested on two tables and the reads are working fine. The resultant tables are same as what spark would produce.

[yankevn]

LGTM