Releases: bgpkit/bgpkit-commons
v0.10.3
New features
- Added RPKISPOOL data source support (draft-snijders-rpkispool-format)
- Parses CCR (draft-ietf-sidrops-rpki-ccr) files from RPKISPOOL
.tar.zstarchives - Uses
bcderfor DER parsing instead of processing ~942K individual.roafiles - Added
RpkiSpoolsCollectorenum with three mirrors:SobornostNet,AttnJp,KerfuffleNet - Public API:
parse_ccr(),parse_rpkispools_archive(),RpkiTrie::from_rpkispools() - Integrated into
BgpkitCommons::load_rpki_historical(),load_rpki_from_files(),list_rpki_files() - Added
rpkispoolsexample
- Parses CCR (draft-ietf-sidrops-rpki-ccr) files from RPKISPOOL
Bug fixes
- Fixed typo in
RpkiViewsCollectorenum variant:SoborostNetrenamed toSobornostNet
Feature flags
- Added
bcderandzstdas optional dependencies under therpkifeature flag
Documentation
- Rewrote
README.mdindependently fromlib.rsdocstrings; README now includes a Mermaid architecture diagram and module summary table - Removed
cargo-readmeworkflow step andreadme.tpltemplate - Added
CLAUDE.mdto.gitignore
v0.10.2
Bug fixes
- Fixed RPKI validation bug where max_length exceeded returned Unknown instead of Invalid
- Prefixes covered by an ROA but exceeding the max_length restriction now correctly return
Invalid - Added
lookup_covering_roas()helper method to find all covering ROAs without max_length filter - Fixed both
validate()andvalidate_check_expiry()methods - Added comprehensive tests for the bug scenarios
- Prefixes covered by an ROA but exceeding the max_length restriction now correctly return
Code improvements
- Replaced
reqwestwithoneiofor RPKIviews file fetching- Simplifies HTTP handling by using the existing
oneiodependency - Maintains streaming optimization for .tgz archives
- Simplifies HTTP handling by using the existing
Maintenance
- Fixed test failures related to AS info name matching
- Updated
list_aspasexample date range to 2023-2025
v0.10.1
Dependency update
- switch
reqwestdefualt ssl backend to rustls
v0.10.0
RPKIviews Historical Data Support
-
Added RPKIviews as a historical RPKI data source: Users can now load historical RPKI data from RPKIviews collectors in addition to RIPE NCC archives
- New
RpkiViewsCollectorenum with four collectors: SoborostNet (default), MassarsNet, AttnJp, and KerfuffleNet - Added
RpkiTrie::from_rpkiviews(collector, date)method for loading from a specific collector - Added
RpkiTrie::from_rpkiviews_file(url, date)andfrom_rpkiviews_files(urls, date)for loading from specific archive URLs - Added
list_rpkiviews_files(collector, date)function to discover available archives for a given date - New
HistoricalRpkiSourceenum to explicitly select between RIPE and RPKIviews sources
- New
-
Streaming optimization for .tgz archives: RPKIviews archives are streamed efficiently without downloading the entire file
rpki-client.jsonis located at position 3-4 in the archive, allowing early termination after ~80MB instead of downloading 300+ MB- New
extract_file_from_tgz(url, target_path)function for streaming extraction of specific files - New
list_files_in_tgz(url, max_entries)function for listing archive contents with early termination - New
tgz_contains_file(url, target_path)function for checking file existence - Uses
reqwestfor HTTP streaming and externalgunzipfor decompression - Test completion time reduced from several minutes to ~8 seconds
-
Unified rpki-client JSON parsing: Extracted shared parsing logic for rpki-client JSON format
- New internal
rpki_client.rsmodule withRpkiClientDatastruct and robust deserializers - Handles variations in ASN formats (numeric
12345vs string"AS12345") - Handles variations in ASPA field names (
customer_asidvscustomer) - Handles provider arrays as both numbers and strings
- Used by Cloudflare, RIPE historical, and RPKIviews sources
- New internal
-
Public ROA and ASPA structs: Added stable public API types
- New
Roastruct with fields:prefix,asn,max_length,not_before,not_after - New
Aspastruct with fields:customer_asn,providers - Internal rpki-client format structs are now
pub(crate)only
- New
-
Updated RIPE historical to use JSON format: Changed from CSV to
output.json.xzfor consistency- Requires
xzfeature in oneio (now enabled by default for rpki feature) - Provides richer data including expiry timestamps
- Requires
-
New BgpkitCommons methods:
load_rpki_historical(source, date)- Load historical RPKI data from specified sourcelist_rpki_files(source, date)- List available RPKI files for a date from specified sourceload_rpki_from_files(urls, date)- Load and merge RPKI data from multiple file URLs
-
New example: Added
examples/rpki_historical.rsdemonstrating historical RPKI data loading -
Updated example:
examples/list_aspas.rsnow counts ASPA objects for first day of years 2020-2025
Dependencies
- Added
reqwest(with blocking feature) for HTTP streaming - Added
tarcrate for reading tar archives - Enabled
xzfeature inoneiofor RIPE historical JSON support
Crate Consolidation
-
Migrated
as2org-rsinto bgpkit-commons: The CAIDA AS-to-Organization mapping functionality previously provided by the externalas2org-rscrate has been fully integrated into theasinfomodule- New
src/asinfo/as2org.rsmodule providesAs2orgstruct withnew(),get_as_info(),get_siblings(), andare_siblings()methods - Removed external
as2org-rsdependency from Cargo.toml - Single codebase simplifies maintenance and patch application
- New
-
Migrated
peeringdb-rsinto bgpkit-commons: The PeeringDB API access functionality previously provided by the externalpeeringdb-rscrate has been fully integrated into theasinfomodule- Updated
src/asinfo/peeringdb.rswith full PeeringDB API client implementation - Includes
PeeringdbNetstruct andload_peeringdb_net()function for direct API access - Removed external
peeringdb-rsdependency from Cargo.toml
- Updated
-
Updated feature flags: The
asinfofeature now usesregexinstead of external crate dependencies- Before:
asinfo = ["as2org-rs", "peeringdb-rs", "oneio", "serde_json", "tracing", "chrono"] - After:
asinfo = ["oneio", "serde_json", "tracing", "chrono", "regex"]
- Before:
API Improvements
- AsInfoBuilder: Added a new builder pattern for loading AS information with specific data sources
- New
AsInfoBuilderstruct with fluent API methods:with_as2org(),with_population(),with_hegemony(),with_peeringdb(),with_all() - Added
asinfo_builder()method toBgpkitCommonsfor creating builders - Added
load_asinfo_with(builder)method toBgpkitCommonsfor loading with builder configuration - The existing
load_asinfo(bool, bool, bool, bool)method is preserved for backward compatibility
- New
Before (confusing boolean parameters):
commons.load_asinfo(true, false, true, false)?;After (clear builder pattern):
let builder = commons.asinfo_builder()
.with_as2org()
.with_hegemony();
commons.load_asinfo_with(builder)?;Public API Enhancements
- asinfo module: Added
PeeringdbDatato public exports for direct module access - All modules now consistently support both:
- Central access via
BgpkitCommonsinstance - Direct module access (e.g.,
bgpkit_commons::bogons::Bogons::new())
- Central access via
Testing Improvements
-
Comprehensive as2org module tests: Added extensive unit tests for the migrated CAIDA AS-to-Organization functionality
- JSON deserialization tests for
As2orgJsonOrgandAs2orgJsonAsstructures - Tests for optional fields and default values
As2orgAsInfostruct creation and serialization round-trip testsfix_latin1_misinterpretationfunction tests for edge cases- Integration tests (ignored by default) for
As2org::new(),get_as_info(),get_siblings(), andare_siblings()methods
- JSON deserialization tests for
-
Comprehensive peeringdb module tests: Added extensive unit tests for the migrated PeeringDB functionality
PeeringdbDatastruct creation, serialization, and deserialization testsPeeringdbNetstruct tests with all optional fieldsPeeringdbNetResponseAPI response deserialization testsPeeringdbstruct tests forget_data(),contains(),len(),is_empty(), andget_all_asns()methods- Empty database edge case tests
- Integration tests (ignored by default) for live API access
-
New Peeringdb helper methods: Added utility methods to the
Peeringdbstruct for better usabilitylen(): Get the number of networks in the databaseis_empty(): Check if the database is emptycontains(asn): Check if an ASN exists in PeeringDBget_all_asns(): Get all ASNs in the database
v0.9.6
Maintenance
- Update dependencies to better handle rustls crypto providers:
- peeringdb-rs to 0.1.3
- oneio to 0.20.0
- as2org-rs to 1.1.1
v0.9.5
Fix CAIDA as2org data loading issue
- Update
as2org-rsto1.1.0to fix CAIDA as2org data loading issue.
Maintenance
- Update dependencies
v0.9.4
Hot-fix
- Remove
let-chaincoding style to maintain compatibility with older Rust versions.
v0.9.3
Maintenance
- Updated
peeringdb-rsto0.1.2to address 403 error from PeeringDB API - Updated
oneioto0.19.0
Code quality
- Addressed clippy warnings across the codebase with focus on
asinfomodule. - Refactored
asinfo_are_siblingsto use if-let chains and Option combinators, removing unnecessaryunwrap()calls. - Simplified conditional logic (collapsible if) for better readability and safety.
v0.9.2
Features
- RPKI expiry support: Added support for Cloudflare RPKI ROA expiry timestamps
- Added
expiresfield toCfRoaEntrystructure for Cloudflare RPKI data - ROA expiry timestamps are now mapped to
not_afterfield inRoaEntry - Added
validate_check_expiry()method toRpkiTriefor time-aware validation - Added
rpki_validate_check_expiry()method toBgpkitCommonsfor expiry-aware validation - Expired or not-yet-valid ROAs now return
Unknowninstead ofInvalid(correct RPKI behavior)
- Added
Bug fixes
- Fixed typo in
rpki_validate()method (vapidate→validate)
Documentation
- RPKI documentation: Added module documentation covering:
- Data structures and validation process explanation
- Usage examples for both real-time (Cloudflare) and historical (RIPE) data sources
- Performance considerations and error handling guidance
- Multiple ROAs per prefix handling examples
- Added
no_runattribute to RPKI documentation examples to prevent timeouts during doc tests
Testing
- Added unit tests for expiry checking functionality
- Added manual integration test for Cloudflare RPKI data loading with expiry validation
- Run with:
cargo test --release --features rpki test_cloudflare_rpki_expiry_loading -- --ignored --nocapture
- Run with:
v0.9.1
Dependencies
- Made
serdea required dependency (no longer optional) to ensure all public types can be serialized/deserialized - Added
SerializeandDeserializederives to all public structs that were missing them