How US analysts mine SEC EDGAR filings for competitive intelligence dashboards

What to take away

• An intelligence dashboard for US competitors runs on EDGAR forms 10-K, 10-Q, 8-K, DEF 14A and S-1, not on scraped press releases.
• XBRL tags from the US GAAP taxonomy give you machine-readable revenue, margin and share counts; the SEC publishes the taxonomy itself.
• CIK lookup ties a company name to every filing it has ever made; the SIC code list groups those companies into peer sets.
• Restatements and fiscal-year shifts break naive time series, so store as-reported and restated values side by side.
• EDGAR bulk data and the Public Dissemination Service move the same facts at lower cost than per-request scraping.
• Every number on the dashboard should trace to a filing accession number a reviewer can open.

Which EDGAR forms carry competitor revenue, headcount and margin signals

Start with the annual report on Form 10-K. It holds audited revenue, cost of revenue, operating income and segment breakouts. Item 6 and the segment note give you the margin lines a competitor revenue dashboard needs. Item 1 gives the business description you can mine for headcount language and plant counts.

Quarterly reports on Form 10-Q keep the series current between annuals. Revenue and margin arrive unaudited but tagged, so they load into the same pipeline. Watch the seasonal pattern: US retailers in Texas and Georgia post a different quarterly shape than Washington software firms.

Current reports on Form 8-K carry the events that move a peer's numbers: acquisitions, divestitures, executive departures, guidance withdrawals. The earnings release attached as Exhibit 99.1 often reaches the wire before the 10-Q is filed.

Proxy statements on Schedule 14A supply headcount proxies. Named executive counts, option grants and segment leadership tell you where a competitor is investing. Registration statements on Form S-1 add pre-IPO peers before they have a 10-K history.

Foreign private issuers file 20-F and 6-K instead, which changes the tag set. If your peer set spans both, normalize the form type before you normalize the values. This is the same discipline behind solid business intelligence examples: define the artifact before you chart it.

XBRL taxonomies and tags worth pulling for a metrics layer

Since the SEC phased in structured data, most large US filers tag their financial statements in XBRL. The tags come from taxonomies the SEC maintains and publishes, and you should pull the current US GAAP taxonomy rather than guessing element names from a filing viewer.

The SEC's taxonomies and schemas are the authoritative source for element definitions and relationships. Version your tag map to the taxonomy year, because elements are deprecated and replaced over time.

Core tags for a competitor revenue dashboard:

• us-gaap:Revenues or us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax
• us-gaap:CostOfRevenue and us-gaap:GrossProfit
• us-gaap:OperatingIncomeLoss
• us-gaap:NetIncomeLoss
• us-gaap:ResearchAndDevelopmentExpense
• us-gaap:Assets and us-gaap:Liabilities
• dei:EntityCommonStockSharesOutstanding

Headcount is harder. There is no single required tag, so pull the textual employee disclosure from Item 1 and the 10-K cover, then store it as a parsed field with a source note. Some filers tag it; most do not.

Segment tags carry a dimension member, so a single revenue tag can appear many times. Filter on the segment axis before you sum, or you will double count. The taxonomy browser shows which axes apply.

Mapping companies with CIK lookup and SIC codes

Every EDGAR filer has a Central Index Key, a ten-digit number that never changes. Use CIK lookup to resolve a company name to its key, then pull every filing under that key. Names collide and subsidiaries file separately, so the CIK is the join key your warehouse should store.

When you resolve a name, capture the former names too. US companies rebrand, and a competitor you track as one entity may have filed under two or three names across a decade. The lookup shows the history.

The SIC code list assigns each filer a four-digit code. Use it to build peer sets: all SIC 7372 software publishers, all SIC 3674 semiconductor makers, all SIC 6021 national banks. Codes are coarse, so treat them as a first cut, not the final peer definition.

Combine CIK and SIC in a mapping table with these columns:

Column	Example	Purpose
cik	0000320193	Join key to filings
sic	3571	Peer grouping
ticker	AAPL	Market data join
name_current	Current legal name	Display
name_prior	Prior names	Historical match
fiscal_year_end	0930	Period alignment

Keep the table under version control. When a competitor acquires a peer, the CIK mapping changes, and your dashboard should show the discontinuity rather than hide it. The same care applies to any dashboards framework you adopt.

Normalizing filings across fiscal years and restatements

Fiscal years do not align. A retailer ending in January, a software firm ending in June and a bank ending in December all report periods of different length. Convert every figure to a common period basis before you chart it.

Restatements are the second trap. A competitor may restate prior revenue after an accounting change or a divestiture. If you overwrite the old value, your trend line lies. Store both the as-first-reported and the restated value, with the accession number and filing date for each.

Steps for a defensible normalization pass:

1. Pull the XBRL company facts for each CIK and keep the frame and period fields.
2. Flag any fact that appears more than once with different values across filings.
3. Tag each value as as-reported or restated using the filing date.
4. Convert fiscal periods to a common monthly or quarterly grid.
5. Reconcile the total to the sum of segments and log any gap.
6. Freeze the normalized table and record the taxonomy version used.
7. Publish a variance report next to the dashboard.

Currency is mostly a non-issue for US filers, but foreign private issuers report in their home currency. Store the reporting currency and convert with a dated rate, not today's rate.

Charting competitor revenue and margin in a working dashboard

Charts should answer a question, not decorate a page. For a competitor revenue dashboard, four views cover most analyst needs.

A small-multiple line chart of revenue by quarter, one panel per peer, shows trajectory without implying a common scale. A stacked bar of revenue by segment shows mix shift. A margin waterfall from gross to operating shows where a competitor loses money. A scatter of revenue growth against margin change separates share gainers from price cutters.

Worked example. You track three US semiconductor firms under SIC 3674. You pull ten years of 10-K and 10-Q XBRL facts by CIK, normalize to calendar quarters, and flag two restatements in year four. The revenue line shows firm A growing faster than the peer median.

The margin waterfall shows its gross margin flat while operating margin falls, because research and development rose. The scatter confirms share gain bought with spending, not pricing. The dashboard surfaces the R&D tag, the restatement flag and the accession number behind each point.

Keep the number of series per chart low. Five lines is the practical limit before a reader stops distinguishing colors. Push detail into a table below the chart.

Label every axis with the unit and the period basis. A revenue figure without a fiscal-period note invites the wrong comparison. These habits also show up in good dashboards examples, where the chart type matches the question.

Using EDGAR bulk data and the Public Dissemination Service

Per-request scraping of EDGAR will get your IP blocked and your pipeline throttled. The SEC provides bulk archives and a dissemination feed for exactly this workload. Pull the quarterly and daily archives, load them into object storage, and parse locally.

The SEC data library indexes the bulk datasets, including financial statement data sets and filing indexes. Download the financial statement data sets if you want pre-parsed XBRL without building your own extractor. They arrive as zip archives with a documented schema.

The Public Dissemination Service is the real-time feed of accepted filings. Subscribe if you need same-minute awareness of an 8-K; use the bulk archives if a nightly refresh is enough. Most competitive intelligence teams need the nightly path, with the feed reserved for event alerts.

Respect the SEC's access rules: declare a user agent, cap request rates, and cache what you fetch. The bulk route is cheaper and kinder to the source than hammering the search interface.

Keeping an intelligence dashboard defensible to reviewers

A reviewer will ask where a number came from. Build the answer into the data model.

• Every fact carries an accession number and filing date.
• Every fact carries the taxonomy version and tag name.
• Restatements are stored, not overwritten.
• CIK and SIC mappings are versioned.
• Fiscal-period conversions are documented.
• Segment sums reconcile to totals, with gaps logged.
• Access rules for EDGAR are followed and logged.

Lineage is not overhead. It is what separates a dashboard a CFO will cite from a spreadsheet nobody trusts. Governance frameworks from bodies such as DAMA International and the Cloud Security Alliance apply here, as do the data standards published by NIST.

Document your peer-set definition, your tag map and your normalization rules in one place. When a competitor restates, the change log explains the movement before anyone asks.

Teams that do this well treat the dashboard as a product with a release history, which is the theme running through business intelligence 2027 and the reporting discipline behind strong reporting examples.

Common questions

Where do I find a company's filings and financial metrics? Use SEC company search to locate filings by name, ticker or CIK. It returns the filing history you can filter by form type.

How do I map a company name to its EDGAR filings? Use CIK lookup to resolve the name to a Central Index Key. Store the CIK, not the name, as your join key.

Which XBRL tags matter most for competitor metrics? Revenue, cost of revenue, gross profit, operating income, net income and research and development expense. Pull the definitions from the SEC taxonomy pages.

How do I group competitors into peer sets? Start with the SIC code list, then refine with segment mix, size and business model. SIC codes are a first cut only.

Can I get EDGAR data in bulk instead of scraping? Yes. The SEC data library hosts bulk archives, and the Public Dissemination Service carries the real-time feed. Both are preferable to per-request scraping.