Fall back to non-overlay analysis when diff-informed analysis is unavailable#3791
Open
sam-robson wants to merge 1 commit intomainfrom
Open
Fall back to non-overlay analysis when diff-informed analysis is unavailable#3791sam-robson wants to merge 1 commit intomainfrom
sam-robson wants to merge 1 commit intomainfrom
Conversation
sam-robson
force-pushed
the
sam-robson/overlay-fallback
branch
2 times, most recently
from
472336b to
8c82546
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Ensures the CodeQL init step avoids the unsupported “overlay-only” mode by falling back to a full non-overlay analysis when diff-informed analysis was expected but didn’t produce the pr-diff-range.json output.
Changes:
- Add a
hasDiffRangesJsonFile()helper to detect whetherpr-diff-range.jsonwas produced. - In
init-action, revertoverlayDatabaseModetoNonewhen overlay is enabled, diff-informed analysis should have run, but the diff ranges file is missing. - Update the generated
lib/init-action.jsto reflect the TypeScript changes.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/init-action.ts | Adds a guard to revert overlay mode to non-overlay when diff-informed output is missing. |
| src/diff-informed-analysis-utils.ts | Introduces hasDiffRangesJsonFile() helper for detecting persisted diff ranges output. |
| lib/init-action.js | Generated build output reflecting the TypeScript changes (not reviewed). |
sam-robson
force-pushed
the
sam-robson/overlay-fallback
branch
from
8c82546 to
08f4ea7
Compare
redsun82
approved these changes
Mar 31, 2026
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
mbg
changed the title
mbg
requested changes
Mar 31, 2026
| ); | ||
| } | ||
|
|
||
| export function hasDiffRangesJsonFile(): boolean { |
Member
There was a problem hiding this comment.
Minor: Adding a JSDoc comment here would be nice.
Comment on lines
+443
to
+466
| // If overlay is enabled and diff-informed analysis should have run but | ||
| // failed to produce output, revert to non-overlay analysis. Overlay | ||
| // without diff-informed is an untested combination that can produce | ||
| // inaccurate results. | ||
| try { | ||
| if ( | ||
| config.overlayDatabaseMode === OverlayDatabaseMode.Overlay && | ||
| (await shouldPerformDiffInformedAnalysis(codeql, features, logger)) && | ||
| !hasDiffRangesJsonFile() | ||
| ) { | ||
| logger.warning( | ||
| "Diff-informed analysis is not available for this pull request. " + | ||
| `Reverting overlay database mode to ${OverlayDatabaseMode.None}.`, | ||
| ); | ||
| config.overlayDatabaseMode = OverlayDatabaseMode.None; | ||
| } | ||
| } catch (e) { | ||
| logger.warning( | ||
| `Failed to determine diff-informed analysis availability, ` + | ||
| `reverting overlay database mode to ${OverlayDatabaseMode.None}: ${getErrorMessage(e)}`, | ||
| ); | ||
| config.overlayDatabaseMode = OverlayDatabaseMode.None; | ||
| } | ||
|
|
Member
There was a problem hiding this comment.
It would be good to move this out of init-action.ts and into one of the overlay-specific modules if possible.
There's already a bunch of logic around overlay enablement there, so spreading that out across different locations isn't good.
We also have some frameworks for tests/diagnostics/telemetry there that this could factor into. In any case, this should have test coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When overlay analysis is enabled for a PR but diff-informed analysis fails (e.g. deleted branch, API error, too many changed files), the action previously continued with overlay-only analysis. This is an untested combination that can produce inaccurate results. Now we fall back to a full non-overlay analysis instead.
Risk assessment
For internal use only. Please select the risk level of this change:
Which use cases does this change impact?
Workflow types:
dynamicworkflows (Default Setup, Code Quality, ...).Products:
analysis-kinds: code-scanning.analysis-kinds: code-quality.Environments:
github.comand/or GitHub Enterprise Cloud with Data Residency.How did/will you validate this change?
If something goes wrong after this change is released, what are the mitigation and rollback strategies?
How will you know if something goes wrong after this change is released?
Are there any special considerations for merging or releasing this change?
Merge / deployment checklist