feat: search current working directory for config file #1464
Conversation
mkdocs/docs/configuration.md
Outdated
There was a problem hiding this comment.
there are a couple other places where pyiceberg.yaml is referenced in the docs
https://grep.app/search?q=pyiceberg.yaml&filter[repo][0]=apache/iceberg-python&filter[path][0]=mkdocs/docs/
There was a problem hiding this comment.
Nice finds! That grep tool is pretty neat. I just made some corrections to these in 3ebbb8c.
There was a problem hiding this comment.
There was a problem hiding this comment.
I tried adding a test here, but I wonder if there are opportunities to clean it up.
|
There's another test that looks to do something similar to this newer test. Are both needed? iceberg-python/tests/utils/test_config.py Lines 58 to 65 in 0e5086c |
Co-authored-by: Fokko Driesprong <fokko@apache.org>
Thank you! I applied that change! ✅ |
| @@ -28,7 +28,7 @@ hide: | |||
|
|
|||
| There are three ways to pass in configuration: | |||
|
|
|||
| - Using the `~/.pyiceberg.yaml` configuration file | |||
| - Using the `.pyiceberg.yaml` configuration file stored in either the directory specified by the `PYICEBERG_HOME` environment variable, the home directory, or current working directory. | |||
There was a problem hiding this comment.
nit: move the extra info about where the file is located down to L37.
also i think its valuable to include a warning about accidentally checking in secrets with git when using the current working directory
| @@ -49,7 +49,7 @@ catalog: | |||
|
|
|||
| and loaded in python by calling `load_catalog(name="hive")` and `load_catalog(name="rest")`. | |||
|
|
|||
| This information must be placed inside a file called `.pyiceberg.yaml` located either in the `$HOME` or `%USERPROFILE%` directory (depending on whether the operating system is Unix-based or Windows-based, respectively) or in the `$PYICEBERG_HOME` directory (if the corresponding environment variable is set). | |||
| This information must be placed inside a file called `.pyiceberg.yaml` located either in the `$HOME` or `%USERPROFILE%` directory (depending on whether the operating system is Unix-based or Windows-based, respectively), in the current working directory, or in the `$PYICEBERG_HOME` directory (if the corresponding environment variable is set). | |||
There was a problem hiding this comment.
nit: include warning about accidentally checking in secrets with git when using the current working directory
| "config_setup, expected_result", | ||
| [ | ||
| # Validate lookup works with: config > home > cwd | ||
| ( |
There was a problem hiding this comment.
nit: for test readability, use ["PYICEBERG_HOME", "HOME", and "CURRENT"]
and replace both with a list ["HOME", "CURRENT"]
There was a problem hiding this comment.
looks like the parameterize test is testing
- PYICEBERG_HOME
- HOME
- CURRENT
- None
- "both" / ["HOME", "CURRENT"]
i'd add a test for all 3
Resolves #1333
Adds the current working directory to the search path for the
.pyiceberg.yamlfile.As it is now, the file is searched in the following order:
PYICEBERG_HOMEenvironment variableI'm unsure if people would like to have 2 and 3 swapped. In either case, users can still override this with the environment variable.