Skip to content

[FTAB-213] Add an example for using both inbuilt and model backed ml functions - #24

Open
Mika Naylor (autophagy) wants to merge 1 commit into
masterfrom
FTAB-213
Open

Mika Naylor (autophagy) wants to merge 1 commit into
masterfrom
FTAB-213

Conversation

@autophagy

Copy link
Copy Markdown
Member

No description provided.

Copilot AI lite review requested due to automatic review settings September 11, 2026 09:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Correct the formatter issue and the scaler terminology and statistic before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a PyFlink example demonstrating built-in ML transformations and AWS Bedrock model-backed embeddings.

Changes:

  • Demonstrates encoding, scaling, and bucketing functions.
  • Registers a Bedrock connection/model and invokes embeddings.
File summaries
File Summary and findings
examples/example_09_ml_functions.py Adds the ML functions example. Findings: line 75 terminology should use “scalers” (nit, 3 votes); line 81 uses an incorrect standard deviation (moderate, 1 vote); line 94 contains a formatter-incompatible blank line (critical, 1 vote).
Review details

Suppressed comments (1)

examples/example_09_ml_functions.py:81

  • The supplied standard deviation does not match these five prices: the population standard deviation is approximately 149.4813, not 147.0. This makes the ml_standard_scaler output non-unit variance, contrary to the explanation above; pass the actual statistic (or change the explanatory text).
        ml_standard_scaler(col("price"), lit(176.1), lit(147.0)).alias("price_standard"),
  • Files reviewed: 1/1 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread examples/example_09_ml_functions.py
Comment thread examples/example_09_ml_functions.py Outdated

AWS_ACCESS_KEY = ""
AWS_SECRET_KEY = ""
AWS_SESSION_TOKEN = ""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a secret we should move to .env?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I'm not sure - if we just want these to be examples people can look at to get a feel of what needs to be done, then I think keeping it explicit like this is fine. Could load it in from an environment variable, but Im not sure - maybe we can just add a comment here explain that its illustrative and that in production youd want to pull this from an environment variable or secret store?

env.use_catalog(TARGET_CATALOG)
env.use_database(TARGET_DATABASE)

env.execute_sql(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to drop the connection and model to clean things up after the run is finished?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so - the other examples like the creating tables example also dont clean up. I think we intend for these to be more illustrative examples of how to use things rather than self contained tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants