Skip to content

Instantly share code, notes, and snippets.

@a-bruhn

a-bruhn/README.md

Last active May 8, 2025 07:35
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save a-bruhn/f33f4578fb5e7ef2d8cbb6c9b73753be to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save a-bruhn/f33f4578fb5e7ef2d8cbb6c9b73753be to your computer and use it in GitHub Desktop.
Download ZIP
Simpler settings for python
Raw
README.md

Pydantic is a great tool all around and can be used to load and validate app settings. The problem is, that this usually results in a lot of environment variables having to be set, which is suboptimal for kubernetes deployments. See a typical example in this configuration module pydantic_settings_version.py.

Instead we usually want to mount our configurations as secrets and configmaps and merge them all together:

  • configuration.yaml
  • secret.yaml

Unfortunately, pydantic does not offer the same partial-merging capabilities as the config crate in Rust (see settings.rs) out of the box and we would have to implement this ourselves, which is a bit overcomplicating things if we also want to have final override-capabilities using environment variables. Instead, we can use this easy workaround shown in pydantic_dynaconf_version.py using dynaconf, which lets us lazily merge our partial sources in the order we specify, which we can then directly pass to pydantic, thus giving us all the flexibility we want.

Raw
configuration.yaml
app:
host: 0.0.0.0
port: 8080
cache:
username: user
host: valkey
port: 6379
db_name: "0"
Raw
pydantic_dynaconf_version.py
from typing_extensions import Self
from pydantic import BaseModel, ConfigDict, SecretStr
from dynaconf import Dynaconf
class _Setting(BaseModel):
model_config = ConfigDict(
from_attributes=True, alias_generator=lambda s: s.upper(), populate_by_name=True
)
class AppConfiguration(_Setting):
host: str
port: int
secret_key: SecretStr
class CacheConfiguration(_Setting):
username: str
password: SecretStr
host: str
port: int
db_name: str
class Settings(_Setting):
app: AppConfiguration
cache: CacheConfiguration
@classmethod
def load(cls) -> Self:
settings = Dynaconf(
settings_files=["configuration.yaml", "secret.yaml"],
envvar_prefix="MYPROJ",
merge_enabled=True,
nested_separator="__",
)
return cls.model_validate(settings)
Raw
pydantic_settings_version.py
# This version only relies on environment variables
# You can add yaml-sources but for merging multiple partial sources with a
# clean resolution order you have to implement something yourself
from pydantic import BaseModel, SecretStr
from pydantic_settings import (
BaseSettings,
SettingsConfigDict,
)
class AppConfiguration(BaseModel):
host: str
port: int
secret_key: SecretStr
class CacheConfiguration(BaseModel):
username: str
password: SecretStr
host: str
port: int
db_name: str
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_prefix="MYPROJ_",
env_nested_delimiter="__",
)
app: AppConfiguration
cache: CacheConfiguration
Raw
secret.yaml
app:
secret_key: mytestsecret
cache:
password: mytestpwd
Raw
settings.rs
// Rust example using the config crate
use secrecy::SecretString;
use serde::Deserialize;
#[derive(Deserialize, Clone, Debug)]
pub struct AppConfiguration {
pub host: String,
pub port: u16,
pub secret_key: SecretString,
}
#[derive(Deserialize, Clone, Debug)]
pub struct CacheConfiguration {
pub username: String,
pub password: SecretString,
pub host: String,
pub port: u16,
}
#[derive(Deserialize, Clone, Debug)]
pub struct Settings {
pub app: AppConfiguration,
pub cache: CacheConfiguration,
}
impl Settings {
pub fn load() -> Result<Self, config::ConfigError> {
let settings = config::Config::builder()
.add_source(config::File::new(
"configuration.yaml",
config::FileFormat::Yaml,
))
.add_source(config::File::new("secret.yaml", config::FileFormat::Yaml))
.add_source(
config::Environment::with_prefix("MYPROJ")
.prefix_separator("_")
.separator("__"),
)
.build()?;
settings.try_deserialize::<Settings>()
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment