Repository navigation
fix: Use valid Postgres type names for Arrow types in entity dataframe uploads - #7142
Conversation
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #7142 +/- ##
=======================================
Coverage 52.39% 52.40%
=======================================
Files 459 459
Lines 58156 58160 +4
Branches 8572 8574 +2
=======================================
+ Hits 30470 30476 +6
+ Misses 25606 25604 -2
Partials 2080 2080
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
|
The only red job is |
ed60413 to
b809993
Compare
…e uploads arrow_to_pg_type, which builds the CREATE TABLE for a pandas entity dataframe, returned type names Postgres does not have: int8 -> tinyint, binary -> binary, null -> null. It also raised for date32 and decimal columns. Map them to smallint, bytea, text, date and numeric. Signed-off-by: sohailshk <122166523+sohailshk@users.noreply.github.com>
b809993 to
a220e8a
Compare
What this PR does / why we need it:
arrow_to_pg_typebuilds theCREATE TABLEfor a pandas entity dataframe in the Postgres offline store. Some of its names are not Postgres types, and some types are not mapped:tinyint(error: type does not exist)smallintbinary(error: type does not exist)bytea(also large_binary)null(syntax error)textValueError: Unsupported typedateValueError: Unsupported typenumericEach case was checked with
df_to_postgres_tableon a local PostgreSQL 18.4: 5 of 5 fail on master, 5 of 5 create and fill the table with this change (date64 checked only in the unit test). Decimal maps to plainnumericbecause this function only gets the type string, and an unconstrained Postgresnumericstores any decimal value exactly.Adds a parametrized test in
test_type_map.py(7 cases, all fail on master).Which issue(s) this PR fixes:
N/A (no existing issue)
Checks
git commit -s)Testing Strategy
Misc
pytest tests/unit/test_type_map.py: 369 passed.