Hierarchical forecasting notebook #664
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
795b9cf to
163ac6e
Compare
|
/gcbrun |
dfbcafe to
d486fb4
Compare
|
go/gcbrun |
|
running test > 4 days |
|
/gcbrun |
09c92fc to
785399c
Compare
|
View / edit / reply to this conversation on ReviewNB ivanmkc commented on 2022-07-07T00:09:11Z Are all of these optimization_objectives applicable for time-series forecasting? reznitskii commented on 2022-07-15T15:28:44Z Forecasting optimization objectives are given here: https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/train-model#optimization-objectives
Looks like all three of the objectives you have here are valid.
Why does the text say AutoMLTabularTrainingJob but the code says AutoMLForecastingTrainingJob? I think the text needs to be cleaned up, variable optimization_prediction_type is not set in the code, variable column_transformations is called column_specs in the code. ivanmkc commented on 2022-07-18T21:16:14Z Ah yes, it's old text. Will update. |
|
View / edit / reply to this conversation on ReviewNB ivanmkc commented on 2022-07-07T00:09:12Z I'm unsure how to demonstrate window_column, window_stride_length and window_max_count for this use case. reznitskii commented on 2022-07-15T15:35:14Z If I understand correctly, the rolling window feature is distinct and separate from hierarchical forecasting. We could add a demonstration of it to the primary forecasting notebook: https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_tabular_forecasting_batch.ipynb ivanmkc commented on 2022-07-18T21:27:10Z Are they mutually exclusive? reznitskii commented on 2022-07-19T14:37:18Z No, not as far as I know. Above, you were concerned that you didn't know how to demonstrate "window_column, window_stride_length and window_max_count for this use case". I'm saying you don't need to worry about it in the context of this notebook. ivanmkc commented on 2022-07-22T18:26:51Z Got it, thanks! |
|
View / edit / reply to this conversation on ReviewNB ivanmkc commented on 2022-07-07T00:09:13Z Line #28. # hierarchy_temporal_total_weight=_TEST_HIERARCHY_TEMPORAL_TOTAL_WEIGHT, I can't cover anything in one notebook but any suggestions on what to do with the hierarchical temporal parameters? reznitskii commented on 2022-07-15T15:39:13Z Is it possible to demonstrate the full set of hierarchical forecasting features within a single notebook? If not and we choose to demonstrate just reznitskii commented on 2022-07-15T16:00:47Z Is it possible to include a picture that demonstrates the hierarchy we use here? Is "product_type" a subset of "product_category", which is a subset of "product"? ivanmkc commented on 2022-07-18T21:27:58Z I'm not sure how to use
Maybe Andrew Leach can help here. AndrewLeach commented on 2022-07-19T00:05:04Z What hierarchical configuration to use depends on what the goal is. Are there product category level metrics we are inspecting (group weight with product category as group column)? Are there season total metrics we are inspecting (temporal weight with no group columns)? Are we checking the overall bias (group weight with no group columns)? |
|
View / edit / reply to this conversation on ReviewNB ivanmkc commented on 2022-07-07T00:09:14Z I plot some error metrics to show prediction performance.
1) I need suggestions on how to show that hierarchical forecasting is working better than non-hierarchical forecasting.
2) Should I show a hierarchical and non-hierachical model (training both) and compare the error against each other? This may be out-of-scope for the notebook but would be a nice addition to docs. reznitskii commented on 2022-07-15T15:44:31Z 1 - Have you talked to Andrew Leach?
2 - Yes, it would be good to compare the results of the hierarchical and the non-hierarchical model, but can we just load the result of the non-hierarchical model for this dataset from somewhere? I would prefer to avoid creating / training the the non-hierarchical model here. ivanmkc commented on 2022-07-18T21:31:42Z
reznitskii commented on 2022-09-06T21:15:26Z
I think comparing with the ground truth is not so meaningful here, the point is to justify doing model training with the additional hierarchical component. ivanmkc commented on 2022-11-29T22:03:07Z Good point. I will remove the ground truth comparison. ivanmkc commented on 2022-11-29T22:40:30Z I think the comparison is out of scope as we move towards notebooks that focus on showcasing how to use the API. |
This is a hierarchical forecasting notebook demonstrating the use of the new hierarchical forecasting parameters for AutoML Forecasting.
If you are opening a PR for
Official Notebooksunder the notebooks/official folder, follow this mandatory checklist:Official Notebookssection, pointing to the author or the author's team.