Export the all-six NEO consensus as a flat-CSV snapshot#1
Open
rlseaman wants to merge 1 commit into
Open
Conversation
Publish the css_neo_consensus membership as a portable CSV so consumers
that cannot reach this database or the upstream sources can classify
designations as NEOs offline.
Motivating consumer: the CSS reprocessing V&V on sikhote, where JPL
CNEOS and NEOfixer are firewalled and Gizmo is unroutable — a live query
and a local six-source rebuild are both impossible, so the consensus
must arrive as a published snapshot.
- scripts/export_neo_consensus.py: read-only (lib.db, claude_ro).
--mode aliases (default, every designation alias from
v_member_designations + per-source flags) | wide (one row/object);
--min-sources N for consensus strength (1=union .. 6=unanimous).
Publish via the existing scripts/upload_release.sh to the 'latest'
GitHub release; consumer pulls with gh release download.
- docs/neo_consensus.md: "Export bridge" section documenting the
publish/consume flow, plus a follow-on note scoping a per-night
station NEO obs export (obs_sbn) to unlock "missing NEOs"
downstream.
NOTE: authored against the documented v_membership_wide /
v_member_designations schema; not yet executed against the live DB
(no Gizmo access from the authoring host). Run on Gizmo to verify
column/timing before relying on it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Some consumers need the NEO consensus but cannot reach this database or the upstream sources. The motivating case is the CSS reprocessing V&V on sikhote: JPL CNEOS and NEOfixer are firewalled there, and Gizmo (this Postgres host) is not routable — so a live query and a local six-source rebuild are both impossible. Those consumers need the consensus as a published snapshot.
That V&V already classifies each reprocessed night's tracklets against MPC
NEA.txt(the one reachable source, ~99.6% of the consensus) and reports total / new / recovered NEOs. This PR lets it swap that backbone for the true all-six consensus with no code change on its side — only the source file changes.What
scripts/export_neo_consensus.py— read-only (lib.db,claude_ro), dumpscss_neo_consensusto CSV.--mode aliases(default): one row per known designation alias (primary / secondary provisional / permid) fromv_member_designations, each carrying the per-source flags fromv_membership_wide. Maximizes match coverage — an observation reported under any alias resolves.--mode wide: one row per object (primary_desig).--min-sources N: consensus strength (1 = union … 6 = unanimous), default 1.scripts/upload_release.shto thelatestrelease; consumer pulls withgh release download.docs/neo_consensus.md— new "Export bridge" section (publish/consume flow) + a scoped follow-on note.Follow-on (noted, not included)
The downstream V&V can report total/new NEOs but not "missing" NEOs — NEOs the original night reported that the reprocessing dropped — because the archived CSS
.mpcd.mrptcarries temporary IDs, not designations. Closing that needs MPC's obs-DB record of which NEO designations a station reported on a given night (anobs_sbnquery keyed on(stn, obs date)joined tov_member_designations). A second exporter,export_station_night_neos.py, is the natural home; happy to follow up if useful.Verification status
Authored against the documented
v_membership_wide/v_member_designationsschema (column names + then_sourcespredicate verified againstsql/consensus/install.sqlandsql/consensus_audit.sql). Not yet executed against the live DB — no Gizmo access from the authoring host. Please run on Gizmo to confirm columns/timing before relying on it:🤖 Generated with Claude Code