Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update my branch #305

Merged
merged 61 commits into from
Jan 17, 2025
Merged

update my branch #305

merged 61 commits into from
Jan 17, 2025

Conversation

tderk
Copy link
Collaborator

@tderk tderk commented Jan 17, 2025

Description

Please include a summary of the changes and the related issue and context.

Fixes #(issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.

  • Tests you ran

Test Configuration:

  • datavault4dbt-Version:
  • dbt-Version: cloud/core, version number
  • dbt-adapter-Version:

Checklist:

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation or included information that needs updates (e.g. in the Wiki)

tkiehn and others added 30 commits November 19, 2024 08:21
@tkiehn
add macro to process new prejoin list syntax
98d001d
@tkiehn
add process_prejoined_columns macro to top-level stage macro
4b0e02d
@tkiehn
change prejoin-logic to perform less joins
07ec2de
@tkiehn
add check and compilation error if a prejoined column is defined twice
fa4087e
@tkiehn
add amount of extract_columns and aliases to amount-mismatch compilat…
54b8720 
…ion error message
@tkiehn
Merge branch 'main' into 287-feature-extract-multiple-columns-from-on…
582af24 
…e-prejoined-object
@tkiehn
add prejoin with source to processing-macro
60b2a98
@tkirschke
removed unnecessary md file
9f1a06b
@tkirschke
First version of yaml parsing, needs to be turned into macro
d18d28b
@tkirschke
First version of macro, and Control_snap_v0 adjusted to use it.
24ee6ba
@tkiehn
Update yaml_metadata_parser.sql, fix if-nesting, rename metadata_dict
956e71a
@tkirschke
Fixed warnings
3a94499
@tkirschke
Merge branch '181-modify-macros-to-directly-work-with-yaml-metadata' of
f3bc4a7 
https://github.com/ScalefreeCOM/datavault4dbt into 181-modify-macros-to-directly-work-with-yaml-metadata
@tkiehn
move stage_processing_macros.sql into staging folder
3fed963
@tkiehn
change extract_input_columns, process_prejoined_columns. add extract_…
edf3dc4 
…prejoin_column_names

change extract_input_columns to handle the new prejoin list syntax. change process_prejoined_columns to parse the old dict syntax to the new list syntax, merging prejoins with the same conditions. add extract_prejoin_column_names which can handle the new list syntax
@tkiehn
add staging.yml with descriptions of process_prejoined_columns and ex…
da7d000 
…tract_prejoin_column_names
@tkiehn
postgres: modify stage to handle new prejoin syntax and simplify sett…
a970a0a 
…ing of commas for ghost records
@tkiehn
bigquery: stage: implement new prejoin syntax
f8767ae
@tkiehn
databricks: stage: implement new prejoin syntax
90dc5c8
@tkiehn
exasol stage: implement new prejoin syntax
706b2af
@tkiehn
fabric stage: implement new prejoin syntax
20d012c
@tkiehn
oracle stage: implement new prejoin syntax
36617f3
@tkiehn
postgres stage add prepend_generated_by()
7adba69
@tkiehn
redshift stage: implement new prejoin syntax
3070e0a
@tkiehn
snowflake stage: implement new prejoin syntax
ef8b375
@tkiehn
synapse stage: implement new prejoin syntax
e891046
@tkirschke
First batch of modified front-end macros
a065dc1
@tkiehn
synapse stage: remove column name escaping in ghost record macro call
71bfee3
@tkiehn
fabric stage fix escape column names
e683e45
@tkiehn
synapse, fabric stages: fix derived input columns
3a86469
tkirschke and others added 29 commits December 13, 2024 12:31
@tkirschke
Finished yaml implementation, not tested yet
f484bbb
@tkirschke
Fixed returning none for optional parameters
1642057
@tkirschke
Fixed ref satellite parameter
2f9e218
@tkiehn
unify formatting of yaml_metadata_parser calls
5e203ca
@tkirschke
Update hash_standardization.sql
aa0ea49
@tkiehn
Update hash_standardization.sql
1e2f721 
add hashing datatype fix for sha1 on databricks also for multi-active hash standardization
@tkiehn
Move warnings to log and improve content of the messages
de6861f
@tkiehn
add ghost record for DATETIME on BigQuery
ec1366c
@tkiehn
fix oracle ghost records for timestamps
bef34c1
@tkiehn
Merge remote-tracking branch 'origin/issue-295-tkirschke-patch' into …
78af272 
…test
@tkiehn
fix behaviour of derived columns throughout the CTEs for all adapers …
659d8ea 
…except synapse and fabric
@tkiehn
fix fabric ghost records for datetime2
c859311
@tkiehn
Merge branch '287-feature-extract-multiple-columns-from-one-prejoined…
0557fbf 
…-object' into test
@tkiehn
Update oracle stage, remove AS for alias of joined tables
6c380d3
@tkiehn
Update oracle ghost_record_per_datatype, simplify detection of (var)c…
e1703c7 
…har types
@tkiehn
redshift: use qualify statement instead of prep CTEs throughout all m…
c0d40bf 
…acros
@tkiehn
change placeholder alias to actual alias in deduplicated_incoming cte…
7f13d91 
… for redshift eff_sat
@tkiehn
small fixes on exasol stage
d143732
@tkiehn
Update ghost_record_per_datatype.sql, fix databricks hash_default_values
1570800
@tkiehn
Merge pull request #300 from ScalefreeCOM/test
76c5ccf 
Pre-Release 1.8.2
@tkiehn
Merge branch 'main' into fix-databricks-ghost-records-binary-defaults
3f9c9f8
@tkiehn
Merge remote-tracking branch 'origin/main' into 298-bug-redshift-not-…
8eb950c 
…using-qualify-for-latest-records
@tkiehn
Merge pull request #301 from ScalefreeCOM/fix-databricks-ghost-record…
e397ce4 
…s-binary-defaults

fix databricks ghost records for binary datatype
@tkiehn
Merge branch 'main' into 298-bug-redshift-not-using-qualify-for-lates…
261d7ab 
…t-records
@tkiehn
Merge pull request #302 from ScalefreeCOM/298-bug-redshift-not-using-…
27f64ae 
…qualify-for-latest-records

Redshift: Use QUALIFY statement instead of prep-CTEs to enhance performance
@tkirschke
Added optional parameter "union_strategy" and implemented it in all n…
c524a57 
…h_link macros
@tkirschke
Added optional parameter "union_strategy" and implemented it in all n…
72e0085 
…h_link macros
@tkiehn
fix nh_link union strategy condition syntax
78d6364
@tkiehn
Merge pull request #303 from ScalefreeCOM/177-nh-link-add-union_strat…
208139c 
…egy-parameter

177 nh link add union strategy parameter
@tderk tderk merged commit 1874c45 into 124-list-of-columns-for-hashdiff-with-includeexclude-parameters Jan 17, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tderk @tkiehn @tkirschke