[Proposal] Make application configuration easier

JP Schneider john.p.schneider at gmail.com
Thu Jun 5 19:11:22 PDT 2014


One potentially worthy note on this:

Using upstart, we can source as many files in whichever order we like.

However, this could potentially break local dev environments, unless we put
all of the dev-required variables into a sample.env type of file but have
our own two or three env files (one with secrets from the current
s3://mofo-techops/creds/apps/<env>/<appname> ), staging.env, and
production.env.

We need not necessarily wait for Habitat to take a patch.



On Tue, Jun 3, 2014 at 11:58 AM, Kieran Sedgwick <kieran.sedgwick at gmail.com>
wrote:

> One more +1.
>
> This is a tough problem that I saw most of last year while contributing
> with you all. As Ali says, this looks like a long-term solution.
>
> That is all :P
>
>
> On Tue, Jun 3, 2014 at 11:27 AM, JP Schneider <john.p.schneider at gmail.com>
> wrote:
>
>> Sorry for the late reply, I was out yesterday.
>>
>> This would be amazing!  I like Gavin's idea as well.
>>
>>
>> On Mon, Jun 2, 2014 at 7:18 PM, Gavin Suntop <gavin at mozillafoundation.org
>> > wrote:
>>
>>> On the error handling tip: I’ve had positive results using JSON/CSON for
>>> config because you can actually write a schema to validate against.
>>>
>>> It does take more work to maintain the schema, but you get very
>>> comprehensive error messages “for free”. You’ll know that you’re actually
>>> missing property X, not just “something broke”.
>>>
>>>  I took this approach on the CSP Logger Service -
>>> https://github.com/mozilla/csp-logger
>>>
>>> Other niceties of this approach are actual types beyond strings.
>>> Like…arrays! No more splitting strings and wondering if you need quotes,
>>> commas, spaces or whatnot.
>>>
>>> +
>>>
>>> *Gavin Lazar Suntop*
>>>
>>> irc, twitter, github:  *gvn*
>>>
>>> On Jun 2, 2014, at 3:42 PM, David Humphrey <
>>> david.humphrey at senecacollege.ca> wrote:
>>>
>>>  I wonder if habitat needs to get taught to be more verbose in dev
>>> situations?  It could tell you exactly what it's doing on app startup.
>>>
>>> Dave
>>>
>>> On 14-06-02 6:38 PM, Kate Hudson wrote:
>>>
>>> The reason I bring up debugging specifically referencing staging/prod
>>> environments is that I really feel it should be possible for me/others who
>>> are not Jon and JP should ideally be able to debug config issues if
>>> both/either of them are away
>>>
>>>  - K
>>>
>>>  On Jun 2, 2014, at 6:02 PM, Kate Hudson <kate at mozillafoundation.org>
>>> wrote:
>>>
>>>  This sounds really good. As long as docs exist for this and we do some
>>> thinking about how resolving bugs will work I think it would definitely be
>>> an improvement on our process.
>>>
>>>  I would love to see us think about the debugging/error resolution
>>> process for common pitfalls in this system, such as:
>>>
>>>  - How do we debug/see which files actually get loaded, and in what
>>> order?
>>> - What if our secret files on s3 contain a legacy env variable? Would it
>>> overwrite the committed environment files? How would we know?
>>> - Do we still have env.sample (probably not)?
>>> - What if we want to *unset* certain env variables (e.g. DEV) that are
>>> defined in default files
>>>
>>>  - Kate
>>>
>>>  On Jun 2, 2014, at 5:38 PM, Jon Buckley <jon at mozillafoundation.org>
>>> wrote:
>>>
>>> Here’s something that really gets my goat: app configuration
>>>
>>> 1) When you install an application initially, you need to copy the
>>> config file from the defaults to one that you can use with `cp env.dist
>>> .env`.
>>> 2) When you pull down updates from git, and something doesn’t work, you
>>> need to copy the config file from defaults again which blows away your
>>> changes, or you need to pick through the change log and apply the changes
>>> manually.
>>> 3) Configuration only via process env (e.g. heroku) is tedious because
>>> there are so many env variables.
>>> 4) When you make a change to the staging or production configs, there is
>>> absolutely no visibility into who changed what, when, or why. This has
>>> broken services for us more than once.
>>> 5) If you’re doing a deploy that requires a config change at the same
>>> time as a code change, you’ll need JP or I to co-ordinate that.
>>>
>>> Therefore a better application configuration solution should:
>>> 1) Work out of the box
>>> 2) Work when pulling down updates
>>> 3) Work with process.env
>>> 4) Be version controlled, preferably using git
>>> 5) Doesn’t require devops to co-ordinate a simultaneous code + config
>>> deploy
>>>
>>> My preferred solution is the following:
>>> - Modify habitat to support loading multiple environment files in a
>>> specific order: https://github.com/brianloveswords/habitat/pull/11
>>> - Add multiple configuration files to each applications repository -
>>> staging.env, production.env, defaults.env
>>> - Code application to support loading from these new locations in order:
>>>  - process environment
>>>  - .env
>>>  - NODE_ENV.env (so staging.env or production.env)
>>>  - defaults.env
>>> - Document this workflow in each app as “MAINTENANCE.md” or something
>>> similar.
>>>
>>> The new workflow when landing a ticket that changes the application
>>> configuration somehow would be to also make changes to staging.env and
>>> production.env in the same patch. For secrets that cannot be made public we
>>> will continue our current process of putting secrets into a config file on
>>> S3.
>>>
>>> Thoughts?
>>> _______________________________________________
>>> Webmaker-dev mailing list
>>> Webmaker-dev at mozilla.org
>>> https://mail.mozilla.org/listinfo/webmaker-dev
>>>
>>>
>>>  _______________________________________________
>>> Webmaker-dev mailing list
>>> Webmaker-dev at mozilla.org
>>> https://mail.mozilla.org/listinfo/webmaker-dev
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Webmaker-dev mailing listWebmaker-dev at mozilla.orghttps://mail.mozilla.org/listinfo/webmaker-dev
>>>
>>>
>>>  _______________________________________________
>>> Webmaker-dev mailing list
>>> Webmaker-dev at mozilla.org
>>> https://mail.mozilla.org/listinfo/webmaker-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> Webmaker-dev mailing list
>>> Webmaker-dev at mozilla.org
>>> https://mail.mozilla.org/listinfo/webmaker-dev
>>>
>>>
>>
>> _______________________________________________
>> Webmaker-dev mailing list
>> Webmaker-dev at mozilla.org
>> https://mail.mozilla.org/listinfo/webmaker-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/webmaker-dev/attachments/20140605/82a587c5/attachment.html>


More information about the Webmaker-dev mailing list