[FTAB-213] Add an example for using both inbuilt and model backed ml functions - #24
Mika Naylor (autophagy) wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Correct the formatter issue and the scaler terminology and statistic before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a PyFlink example demonstrating built-in ML transformations and AWS Bedrock model-backed embeddings.
Changes:
- Demonstrates encoding, scaling, and bucketing functions.
- Registers a Bedrock connection/model and invokes embeddings.
File summaries
| File | Summary and findings |
|---|---|
examples/example_09_ml_functions.py |
Adds the ML functions example. Findings: line 75 terminology should use “scalers” (nit, 3 votes); line 81 uses an incorrect standard deviation (moderate, 1 vote); line 94 contains a formatter-incompatible blank line (critical, 1 vote). |
Review details
Suppressed comments (1)
examples/example_09_ml_functions.py:81
- The supplied standard deviation does not match these five prices: the population standard deviation is approximately 149.4813, not 147.0. This makes the
ml_standard_scaleroutput non-unit variance, contrary to the explanation above; pass the actual statistic (or change the explanatory text).
ml_standard_scaler(col("price"), lit(176.1), lit(147.0)).alias("price_standard"),
- Files reviewed: 1/1 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
3d27c5b to
3006562
Compare
|
|
||
| AWS_ACCESS_KEY = "" | ||
| AWS_SECRET_KEY = "" | ||
| AWS_SESSION_TOKEN = "" |
There was a problem hiding this comment.
Looks like a secret we should move to .env?
There was a problem hiding this comment.
Hm. I'm not sure - if we just want these to be examples people can look at to get a feel of what needs to be done, then I think keeping it explicit like this is fine. Could load it in from an environment variable, but Im not sure - maybe we can just add a comment here explain that its illustrative and that in production youd want to pull this from an environment variable or secret store?
| env.use_catalog(TARGET_CATALOG) | ||
| env.use_database(TARGET_DATABASE) | ||
|
|
||
| env.execute_sql( |
There was a problem hiding this comment.
Do we need to drop the connection and model to clean things up after the run is finished?
There was a problem hiding this comment.
I don't think so - the other examples like the creating tables example also dont clean up. I think we intend for these to be more illustrative examples of how to use things rather than self contained tests.
No description provided.