Data Methodology & Sources

About the Data

Every number on this platform comes from the U.S. Energy Information Administration's Open Data API — the authoritative source for international energy statistics.

Primary Data Source

U.S. Energy Information Administration (EIA)

The EIA is an independent agency of the U.S. Department of Energy that collects, analyzes, and disseminates energy information. Its Open Data API (v2) provides programmatic access to comprehensive international energy trade statistics spanning decades.

EIA Open Data PortalAPI Documentation

Data Coverage

Temporal and geographic scope

2000 – 2023
Year Range
Annual resolution
10
Countries
Major trading economies
5
Energy Types
Primary categories

Covered Countries

🇺🇸United States(USA)🇨🇳China(CHN)🇮🇳India(IND)🇷🇺Russia(RUS)🇸🇦Saudi Arabia(SAU)🇩🇪Germany(DEU)🇯🇵Japan(JPN)🇨🇦Canada(CAN)🇦🇪UAE(ARE)🇦🇺Australia(AUS)

Energy Type Mapping

EIA activity codes used per energy category

Energy TypeImport ActivityExport ActivityUnit
Petroleum & LiquidsPATIMPPATEXPthousand barrels/day
Crude OilCRNTIMPCRNTEXPthousand barrels/day
Natural GasNGIMPNGEXPbillion cubic feet
CoalCLIMPCLEXPmillion short tons
ElectricityELIMPELEXPbillion kWh

Data Pipeline Methodology

How raw EIA data becomes the charts you see

1
Raw Data Fetch
EIA API v2 is queried for each country–energy-type–direction combination using facet filtering. Requests are rate-limited to 4/sec.
2
Normalization
Raw EIA records are mapped to an internal schema. Suppressed values ("w"), withheld values ("*"), and nulls are replaced with null and flagged for downstream consumers.
3
Aggregation
Multiple series per year are averaged. Values are stored in the EIA-reported unit for each energy type. No currency conversion is performed.
4
Persistence
Normalized data is written to MongoDB Atlas time-series collections via upsert, keyed on (country, energyType, direction, period). Duplicate runs are idempotent.
5
Caching
API responses are cached in Upstash Redis with TTLs of 1–24 hours depending on endpoint. Next.js ISR adds a second cache layer at the CDN level.

Data Quality & Special Values

How missing and suppressed data is handled

"w"
Withheld
Stored as null
"*"
Less than threshold
Stored as null
null
Not reported
Stored as null

Charts skip null data points and display gaps rather than interpolating or filling. This preserves data integrity and prevents misleading visualizations.

Update Frequency

EIA Source Data
Annual (historical), varies for current year
EIA publishes final annual figures ~12 months after year-end
MongoDB (ETL Script)
Manual / scheduled run of fetchEIAData.py
Re-run the Python script to pull updated EIA figures into the database
Redis Cache
1–24 hours TTL per endpoint
API responses are cached to avoid redundant DB queries
Dashboard UI
On filter change
RTK Query revalidates when cache TTL expires

Disclaimers

  • This platform is an independent analytics tool and is not affiliated with or endorsed by the U.S. Energy Information Administration.
  • Data accuracy depends on EIA source quality. Small discrepancies from official EIA publications may exist due to rounding or API versioning.
  • Forecasts shown on the Trends page are simple linear regressions for illustrative purposes only — not investment or policy guidance.
  • Trade balance figures are calculated as (exports − imports) within each energy type and country. Cross-commodity aggregation is intentionally not performed.