Mailchimp safe 1169

[ddfridley]

Problem

We were having trouble with bot emails ending up in mailchimp, and we are closing vulnerability signup and newsletter opt-in flow.

• Users could be sent to Mailchimp before verifying their DemocracyLab email address.
• Signup lacked strong server-side bot protections.
• reCAPTCHA requests were blocked by CSP in dev, which could prevent successful signup.
• The footer newsletter subscribe button pointed to an invalid Mailchimp form URL.

In addtion:

• Docker builds failed on fresh machines because of an outdated Yarn apt signing key.

What This PR Changes

• Defers Mailchimp subscription until after DemocracyLab email verification succeeds.
• Adds server-side signup abuse controls:
  • reCAPTCHA token verification
  • IP-based signup rate limiting
• Updates Mailchimp member status to subscribed after app-level verification to avoid a redundant second confirmation email.
• Replaces the broken footer newsletter endpoint with the current hosted Mailchimp signup URL.
• Adds required CSP connect-src entries for reCAPTCHA endpoints and documents why.
• Adds Mailchimp setup guidance in code to reduce future configuration drift.

Dev Server Testing

On the dev server we had to set

• MAILCHIMP_API_KEY - to a new key because the old one was shutdown.
• MAILCHIMP_SUBSCRIBE_LIST_ID - need to make sure what's in production is this.
• FAKE_EMAILS to False to generate emails,
• REDIS_ENABLED to False because no worker is being started in dev to process the request queue.

[ddfridley]

Claude generated the change to Dockerfile - and I had Gemini Pro review it for security since it's pretty opaque to me. But I could get docker to build on a fresh machine without this.

I have tested user signup on the dev server and it works and doesn't add users to mailchimp until after they verify their email.

[ddfridley]

I added common/static to .gitignore because in my other branches webpack and storybook are generating lots of files there, but these files shouldn't be added to the repo.

[marlonkeating]

Reviewed with help of Claude.

One thing I found during testing on the dev environment, registering threw an error that looks related to database migration when I attempted to sign up:
Jun 28 09:06:36 AM democracy-lab-dev app/web.1 django.db.utils.ProgrammingError: column "newsletter_signup_requested" of relation "democracylab_contributor" does not exist

[ddfridley]

@marlonkeating

Reviewed with help of Claude.

One thing I found during testing on the dev environment, registering threw an error that looks related to database migration when I attempted to sign up: Jun 28 09:06:36 AM democracy-lab-dev app/web.1 django.db.utils.ProgrammingError: column "newsletter_signup_requested" of relation "democracylab_contributor" does not exist

I had this problem too, until I ran heroku run python manage.py migrate democracylab but since I ran it, it shouldn't have to be run again. Is there some other server accessing the same database?

Claude is saying: "his error is happening because the database schema on democracy-lab-dev is out-of-sync and the migration wasn't physically applied to that database during the deploy. We can fix the dev environment by running heroku run python manage.py migrate democracylab or by temporarily reverting and re-applying the migration history on that shared DB."

I'm looking into it, but wondering if sharing the same DB with some other server?
Thanks,

[ddfridley]

The fix for the migration issues was to auto generate migrations for the change to Django 4. The logs show the migrations working now. See those files for a more detailed explanation.

I have pushed these changes dev and it's working.