Dev #219

DamianHinz · 2024-08-08T06:59:33Z

No description provided.

…se call

Merge parameter fixes

- Add a ldts_default_dtype to the dbt_project - Cast ldts to the ldts_default_dtype only for bigquery

use the macro to set a default timestamp data type for staging tables which casts the ldts to the default timestamp data type

…amp_default_dtype cast src_ldts to the global variable timestamp_default_dtype

…rdise macro concattenated_standardise does not take "is_hashdiff" and "rtrim_hashdiff", therefore the parameters have to be removed from the macro call

…f from concattenated_standardise parameters "is_hashdiff" and "rtrim_hashdiff" are not used in the concattenated_standardise macro, therefore they can be removed from the macro parameters

Merge Exasol hotfix rec track sat fixes into dev-branch

…default_dtype global vars to dictionaries macro contains logic to parse global variable dictionary and return string types for each supported adapter based on the type(rsrc, stg, derived_columns) different variables from dbt_project.yml are processed therefore different types can be set

…sing-macro

fix typos

Add type to warning-message

Add hashkeys to processed_hash_columns dict so the ghost records are created for them

UNION expects Keyword ALL or DISTINCT, but got keyword SELECT add a UNION DISTINCT

Add lower-filter to columns_without_excluded_columns-creation for comparison with derived_column_names Comment out line that included derived_column_names in final_columns_to_select

change hardcoded default hashdatatype to VARCHAR(32)

…macro

…DATEADD for redshift

…tandardise because string_agg is not suooported in redshift

…parison with derived_column_names

…ut_excluded_columns to properly exclude derived columns

…cremental is true

…ation If case_sensitive is true there is an UPPER( in the prefix But the amounts of closing brackets was not correct in the suffix

tkiehn and others added 30 commits July 1, 2024 12:08

exasol hub.sql, add disable_hwm parameter

ccb6f40

exasol link.sql, add disable_hwm parameter

757e512

exasol rec_track_sat.sql, add disable_hwm parameter

c90028d

Update hash_standardization.sql, add disable_hwm parameter

047eaee

Update hash.sql, add is_hashdiff parameter to concattenated_standardi…

faf345e

…se call

Update hash_standardization.sql, add , rtrim_hashdiff

651403a

Update hash.sql; fix parameters

27c7f08

postgres sat_v0.sql, add disable_hwm and source_is_single_batch params

db9b7e2

redshift sat_v0.sql, add , disable_hwm and source_is_single_batch params

2632c85

Adjusted source model processing for exasol rec track sat

815364a

Merge pull request #193 from ScalefreeCOM/tkiehn-patch-2

e707c7e

Merge parameter fixes

Exasol hub and Link fixed for multi source processing

e699130

Fixed multi source processing for Synapse

c305fcc

Fixed Multi Source Processing for NH Links (Exasol)

9c9146a

Fixed list recognition for exasol hubs

39c8b0f

Update dbt_project.yml, add default ldts dtype

982545a

Update stage.sql for bigquery, added cast ldts to ldts_default_dtype

b7d77aa

- Add a ldts_default_dtype to the dbt_project - Cast ldts to the ldts_default_dtype only for bigquery

add cast ldts to default_ldts_type to snowflake stage

b360a1f

add timestamp_default_dtype-variable and parsing-macro

e7dc117

use the macro to set a default timestamp data type for staging tables which casts the ldts to the default timestamp data type

Rename default_timestamp_dtype.sql to timestamp_default_dtype.sql

27f95fa

fix macro call

940a5d1

Update rec_track_sat.sql, cast src_ldts to the global variable timest…

0a11b79

…amp_default_dtype cast src_ldts to the global variable timestamp_default_dtype

Update hash.sql, remove hashdiff parameters from concattenated_standa…

49502b5

…rdise macro concattenated_standardise does not take "is_hashdiff" and "rtrim_hashdiff", therefore the parameters have to be removed from the macro call

Update hash_standardization.sql, remove is_hashdiff and rtrim_hashdif…

bd1951b

…f from concattenated_standardise parameters "is_hashdiff" and "rtrim_hashdiff" are not used in the concattenated_standardise macro, therefore they can be removed from the macro parameters

Merge pull request #201 from ScalefreeCOM/exasol-hotfix-rec-track-sat

28e5ad0

Merge Exasol hotfix rec track sat fixes into dev-branch

modify stage, rec_track_sat and derived_columns macros to use new par…

00e020c

…sing-macro

Update string_default_dtype.sql

1e45bdc

fix typos

Update string_default_dtype.sql

5dd89ba

Update string_default_dtype.sql

f19b5b4

Add type to warning-message

tkiehn and others added 29 commits July 11, 2024 10:23

Update bigquery stage.sql

f3c341b

Add hashkeys to processed_hash_columns dict so the ghost records are created for them

Update ref_table.sql, add a UNION DISTINCT for the cte dates

a0634bd

UNION expects Keyword ALL or DISTINCT, but got keyword SELECT add a UNION DISTINCT

Update hash_standardization.sql, fix redshift standardise_suffix

cfed16f

Update redshift stage.sql

ce00d44

Add lower-filter to columns_without_excluded_columns-creation for comparison with derived_column_names Comment out line that included derived_column_names in final_columns_to_select

Update pit.sql

f51cdcb

change hardcoded default hashdatatype to VARCHAR(32)

fix ref key processing for multiple sources

58d670e

change deduplication logic of ref_hub to similar as in the snowflake …

9ed9f1e

…macro

parameter fixes

332567b

change coalesce statement in pit_records from using TIMESTAMP_SUB to …

8db585b

…DATEADD for redshift

remove rtrm_hashdiff parameter

70034c2

change string_agg to listagg in redshift_multi_active_concattenated_s…

db0c13f

…tandardise because string_agg is not suooported in redshift

Update pit.sql, Add cast to for type and rsrc to string_default_dtype

13bd6c9

change interval syntax for postgres control_snap_v1

0ae3f5d

Add lower-filter to columns_without_excluded_columns-creation for com…

d8e0d49

…parison with derived_column_names

change escape_column_names casing to lower for postgres

ad37437

postgres stage: change default hashtype to varchar, fix columns_witho…

70c0d75

…ut_excluded_columns to properly exclude derived columns

change default hash_dtype for postgres pits to varchar(32)

a665667

fix postgres ref-hub with multiple sources

0ea7e2d

fix postgres multi active behavior

a7eb69a

fix nh-link incremental selection for last cte

850afdf

change true to false in log statement

5087084

Update nh_link.sql, use ns.last_cte in records to insert cte if is_in…

a7fd4ff

…cremental is true

Update pit.sql, use cross join instead of full outer join

9a44ff3

Update ref_table.sql, use Cross Join instead of Full Outer Join

9bfd087

Update dbt_project.yml, fix postgres timestamp format

0f2c53a

Update end_of_all_times.sql, add postgres-macro

c9a4bd3

Update timestamp_format.sql, fix postgres timestamp

7af901e

Update datatypes.sql

e170bb6

Update hash_standardization.sql, fix redshift multi_active_standardiz…

701b63a

…ation If case_sensitive is true there is an UPPER( in the prefix But the amounts of closing brackets was not correct in the suffix

DamianHinz merged commit 1963258 into 110-control-who-triggers-actions Aug 8, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev #219

Dev #219

DamianHinz commented Aug 8, 2024

Dev #219

Dev #219

Conversation

DamianHinz commented Aug 8, 2024