Incident Summary
During a production demo, users were unable to sign up or sign in to the application. Firebase Authentication returned a 503 Service Unavailable error when attempting to create new accounts.
Diagnostic Steps
1. Check Firebase Functions Logs
firebase functions:log
This revealed repeated errors from the blocking function with audience claim mismatches.
2. Get Detailed Cloud Functions Logs via gcloud
For 1st gen functions:
gcloud functions logs read --limit=50 --project=PROJECT_ID
For 2nd gen functions (Cloud Run):
gcloud run services logs read FUNCTION_NAME --project=PROJECT_ID --region=REGION
This showed successful executions from previous days, confirming the function code itself was working.
3. List All Deployed Functions
gcloud functions list --project=PROJECT_ID
Check the RUNTIME and TRIGGER columns to verify function generation.
4. Verify Deployment Configuration
firebase deploy --only functions --dry-run
Output confirmed the function was configured as 2nd Gen but revealed an outdated firebase-functions package warning.
5. Check Package Versions
cat functions/package.json | grep firebase
Verified firebase-functions version (should be v4.0+ for proper v2 support).
6. Check Identity Platform Blocking Function Registration
Navigate to Firebase Console > Authentication > Settings > Blocking functions to verify the registered URL format matches the expected generation.
Error Signature
The Firebase Functions logs revealed the following error:
FirebaseAuthError: Firebase Auth Blocking token has incorrect "aud" (audience) claim.
Expected "run.app" but got "https://us-west1-PROJECT_ID.cloudfunctions.net/onUserCreated"
Root Cause
The application uses a beforeUserCreated blocking function to:
- Create user documents in Firestore
- Process pending invitations
- Set initial custom claims
The function was correctly written using Firebase Functions v2 SDK:
const { beforeUserCreated } = require("firebase-functions/v2/identity");
exports.onUserCreated = beforeUserCreated(async (event) => {
// User creation logic
});
However, Firebase Identity Platform had registered the blocking function URL as a 1st generation Cloud Functions endpoint (cloudfunctions.net) instead of the 2nd generation Cloud Run endpoint (run.app).
When Identity Platform invokes a blocking function, it generates a JWT token with an aud (audience) claim. The Firebase Admin SDK validates this claim against the expected Cloud Run URL format. The mismatch between the registered URL and the expected format caused the validation to fail, resulting in the 503 error.
Resolution
-
Redeployed the blocking function to force re-registration:
firebase deploy --only functions:onUserCreated -
Verified the deployment showed “2nd Gen” in the output:
functions[onUserCreated(us-west1)] Successful update operation. -
If the issue persists after redeployment, manually update the blocking function registration:
- Navigate to Firebase Console > Authentication > Settings > Blocking functions
- Remove the existing registration
- The function should auto-register with the correct Cloud Run URL on next deployment
Prevention
-
Verify function generation: After deploying blocking functions, check Firebase Console to confirm the registered URL ends with
run.app, notcloudfunctions.net -
Use consistent SDK versions: Ensure
firebase-functionspackage is v4.0+ for proper v2 function support -
Test blocking functions in emulator first: The Firebase Auth Emulator can catch configuration issues before production deployment
-
Monitor function logs post-deployment: Check for audience claim errors immediately after deploying changes to blocking functions
Technical Details
| Component | Expected | Actual (Broken) |
|---|---|---|
| Function Generation | 2nd Gen (Cloud Run) | 1st Gen (Cloud Functions) |
| URL Pattern | *.run.app | *.cloudfunctions.net |
| Token Audience | Cloud Run URL | Cloud Functions URL |
The Firebase Admin SDK’s FirebaseTokenVerifier performs strict audience validation. When the Identity Platform sends a token with a cloudfunctions.net audience to a function expecting run.app, the verification fails with auth/argument-error.