Release notes#

PyStarburst 0.9.0 - July 08, 2024#

Features#

  • Add session.cache_results method that allows caching dataframes

  • Add df.write.copy_into_location method that allows export dataframe to external location

Dependencies#

  • Bump pydantic to v2 and adjust code to still use pydantic v1

  • Bump dependencies and trino-python-client to 0.329.0

PyStarburst 0.8.0 - April 30, 2024#

Breaking Changes#

  • Use JodaTime’s DateTimeFormat. Previously we used Teradata functions underneath,
    which for date and time year-month-day hour:minute:second use format:
    yyyy-mm-dd hh24:mi:ss
    Now we use JodaTime’s DateTimeFormat, which use format:
    yyyy-MM-dd HH:mm:ss
    which is far more common. Going forward please use JodaTime’s DateTimeFormat in to_date and to_timestamp functions.

Features#

  • Add explode, posexplode, inline dataframe methods

  • Add struct function

  • Support schema discovery for Iceberg and Delta

  • Add if_exists clause to Table.drop_table

Fixes#

  • Do not accept a boolean value for ‘on’ parameter in ‘join’ method

Under the Hood#

  • Compress logical plan if server supports it

Dependencies#

  • Update trino-python-client to 0.328.0

  • Update dependencies

PyStarburst 0.7.0 - February 21, 2024#

Features#

  • Add DataFrame.is_empty() method

  • Adds melt alias to unpivot

  • Add DataFrame.to() method

  • Add DataFrame.with_columns_renamed() and modified with_column_renamed accordingly

  • Add LambdaFunctionExpression

  • Add transform function

  • Add zip_with function

  • Added filter function

  • Added any_match function

  • Added all_match function

  • Added array_sort function

  • Added reduce function

  • Added transform_keys, transform_values functions

  • Added map_filter function

  • Added map_zip_with function

  • Support type coercion

  • Add json_array_length function

  • Add is_not_null function

  • Add to_json function

  • Support type coercion

  • Add from_json function

  • Add programmatic access to Starburst schema discovery feature, using discover method

  • Add get_json_object function

  • Add json_tuple function

Under the Hood#

  • Use EXECUTE IMMEDIATE for prepared statements by default

  • Allow up to 3 arguments for LambdaFunctionExpression

  • Give random names for lambda parameters

  • Add tests against Python 3.12

  • Lambda functions retain context

  • Remove source_plan from TrinoPlan

Dependencies#

  • Remove typing-extensions dependency

  • Remove backports-zoneinfo dependency

  • Remove testcontainers dependency

  • Update dependencies

  • Update trino to 0.327.0

PyStarburst 0.6.3 - November 09, 2023#

Under the Hood#

  • Conda release

PyStarburst 0.6.2 - October 18, 2023#

Features#

  • Add support for stack function

  • Implement to_pandas() method

  • Add map functions: create_map, map_from_arrays, map_keys, map_values, map_entries, map_from_entries, map_concat

  • Add array functions: reverse, flatten, sort_array, shuffle, sequence, array_prepend, array_compact, get

  • Add support for unpivot function

Fixes#

  • Fix to_timestamp to accept not only VARCHAR as an argument

Under the Hood#

  • PyPi release

PyStarburst 0.5.0 - August 17, 2023#

Features#

  • Add support for regexp_split

  • Add collect_list and collect_set

  • Add support for TIME WITH TIME ZONE

  • Implement array functions: array_distinct, array_min, array_max, array_union, array_except, array_slice, size, array_repeat, array_insert, array_join, array_remove, element_at, arrays_zip

  • Support for aggregate functions: max_by, min_by, product

  • Support tuple as struct

  • Support ilike column function

  • Support math functions