Building a Knowledge Graph from Extracted Web Data

Google's Knowledge Graph contained roughly 54 billion entities and 1.6 trillion facts as of early 2024, drawn from Wikipedia, Wikidata, the CIA World Factbook, Google Books, structured website markup, and domain partners. The number itself is less instructive than what sustaining it demands of pipeline design.

In July 2023, Google added over 10 billion new entities in a matter of days. In March 2024, roughly 4 billion entities were added in a single day. No manual review process survives contact with those ingestion rates. Every stage, extraction, entity recognition, resolution, schema alignment, loading, must be automated end-to-end in what amounts to a continuous ETL pipeline, because the moment a human is required in the critical path of a single entity's ingestion, the pipeline cannot operate at this tempo.

Wikidata offers a useful counterpoint. As of 2025, it contains more than 100 million entities, over 12,000 properties, and approximately 10 billion triples, grown through a combination of human editors and automated bots. It is slower and more carefully curated than Google's graph, and that tradeoff surfaces visibly in data quality and governance. Automation produces volume; curation produces precision. Production systems must decide, clearly and early, where on that spectrum they need to sit, because the architecture required for each end is markedly different.

Diffbot's commercially oriented knowledge graph, built entirely from web extraction, organizes close to 10 billion linked datasets across companies, products, articles, and discussions. It is the most practically useful reference for teams building domain-specific graphs without Google's infrastructure or Wikidata's volunteer community. What it demonstrates is that the pipeline works at commercially meaningful scale when each stage is implemented with genuine rigor, where rigor means the graph corrects its errors rather than quietly compounding them.

These three examples are not aspirational benchmarks. They are evidence of what deliberate pipeline design produces and, by negative implication, what the absence of it costs. A brittle graph does not typically collapse at scale. It collapses long before getting there, usually at the resolution stage, and the collapse is quiet: the graph grows, queries return results, and the results are wrong.

Stage One: Getting Usable Raw Material Out of the Web

The first problem in web extraction is that the web is no longer mostly static HTML. JavaScript-heavy single-page applications render their content client-side via dynamic rendering, which means a plain HTTP request returns a shell with no data. Headless browser adoption among scraping practitioners rose from 35% in 2021 to 58% in 2024, per data cited by Scrappey drawing on Gitnux figures. That shift reflects something real: the majority of pages worth extracting now require a full browser rendering pipeline just to surface the content.

Rule-based extraction, writing CSS selectors or XPath expressions to target specific elements rather than relying on schema.org structured data markup, works until the page layout changes. Layout changes constantly. A selector that reliably extracts a product price in January may return nothing by March, or return the wrong element entirely. The maintenance burden compounds silently, and the engineers who wrote the original selectors are rarely the ones who discover the breakage months later.

AI-native extraction changes the abstraction level. Instead of specifying where data is on a page, you describe what data you need, and the model interprets the structure. Per Browserless's state-of-scraping report, this intent-based approach represents the most significant recent reduction in the need for specialized scraping knowledge. A model that understands what a product name is will locate it under a new layout more reliably than a selector targeting a specific CSS class. But what if the layout is sufficiently novel that even semantic interpretation breaks down? The failure modes are different rather than absent: sufficiently novel layouts can still confound semantic interpretation, and when they do, the failure is harder to anticipate than a broken selector.

Two outputs are required from this stage before anything else matters: clean, consistently encoded text that NLP processing can handle, and provenance metadata, meaning source URL, timestamp, and page type at minimum. Provenance is not housekeeping. It is the mechanism by which entity resolution and graph maintenance become tractable later. Without it, a wrong fact has no traceable origin.

The common failure modes are predictable. Anti-bot measures and rate limiting block bulk collection at scale. Boilerplate content, navigation menus, footer links, sidebar ads, passes downstream as substantive text, and the NLP stage will extract entities from it faithfully. Skipping JavaScript rendering leaves dynamic content invisible to the pipeline entirely. Tools like Firecrawl and Diffbot address different points along this spectrum; neither eliminates the core constraint. Garbage introduced at this stage does not announce itself. It surfaces as a wrong node three stages later.

Stage Two: Pulling Entities and Relationships Out of Unstructured Text

Given clean text, the extraction stage's job is to identify nodes and edges for the graph: named entity recognition, or NER, surfaces the nodes, relation extraction identifies the edges, and attribute extraction populates the properties on both.

The tooling for this task has passed through three discernible phases. First came handcrafted linguistic rules and pattern matching: interpretable, auditable, fast to run, brittle in direct proportion to the specificity of the patterns. Then came deep learning, particularly BiLSTM-CRF architectures and later Transformer-based models, which shifted the work from pattern authorship to annotated data curation. The third phase, where most production systems now operate, uses large language models directly for extraction.

What LLMs change in practice is concrete. Pre-labeled training corpora for each new domain are no longer strictly required. A model with sufficient language understanding can extract entities and relationships from an unfamiliar domain with no fine-tuning, a capability commonly called zero-shot extraction, given clear instructions. More importantly, it captures relationships that surface-pattern systems miss because those relationships are expressed through semantic content rather than syntactic structure. Consider: "Following the departure of its longtime chief executive, the board named her successor from within the finance division." That sentence contains an employment relationship, a departure event, and a succession relationship. A rule-based system targeting verb phrases around "employed by" misses all three.

A 2025 academic study on knowledge graph construction, available at arxiv.org/pdf/2502.14192, extracted 620,353 entities and 2,271,584 relations from a research corpus. Manual sampling of 100 papers showed entity extraction accuracy of 0.94 and inter-paper relationship accuracy of 0.93. Those numbers are strong relative to earlier approaches. But why exactly does this happen to matter so much at scale? Accuracy in the low-to-mid 90s across millions of relations means a non-trivial absolute count of wrong edges enters the graph, and that count grows linearly with scale. This shapes what the resolution and validation stages downstream must absorb.

LangChain's LLMGraphTransformer was among the first widely adopted implementations of this approach. GLiNER, Relik, and KGGen are now in active use alongside it, each with different tradeoffs in speed, accuracy, and the degree to which they require predefined entity types. Microsoft GraphRAG's extraction pipeline offers a concrete end-to-end example: GPT-4 extracts entities with specialized prompts, relationships are mapped through co-occurrence and semantic similarity, and the Leiden algorithm is applied for hierarchical community detection. The sequence treats extraction as a multi-pass process rather than a single inference step, which is consistent with how production systems handle precision requirements at scale.

Stage Three: Entity Resolution and Why Skipping It Produces a Broken Graph

Here is what a knowledge graph actually looks like when entity resolution is skipped. "Apple Inc.," "Apple," and "AAPL" each become separate nodes. Queries for facts about the company return partial results depending on which surface form appears in any given source. Paths between entities that should be connected are broken because one leg of the path uses one surface form and another leg uses a different one. The graph is technically loaded, technically queryable, and wrong in ways that are difficult to detect without ground-truth validation.

The relation side of the problem is less obvious but equally corrosive. Raw extraction produces an enormous variety of predicate forms. "Works at," "is employed by," "holds a position at," "serves as," and "joined" may all encode the same relationship between a person and an organization. Without normalization, each becomes a distinct edge type. A query for all employees of a given company must then enumerate every predicate variant or miss the results expressed in forms it fails to enumerate. The graph contains the information and cannot reliably surface it, which is arguably worse than not having the information: it produces false confidence.

Entity resolution involves three distinct operations, each of which must execute before the graph is populated. Entity clustering, sometimes called deduplication, groups surface variants that refer to the same real-world instance, using a combination of string similarity, embedding-based semantic similarity, and contextual signals from surrounding text. Relation normalization collapses semantically equivalent predicates into a single canonical form. Coreference resolution links pronouns and noun phrases within individual documents back to their named referents, so that "she," in a document about a particular executive, resolves to that executive's entity rather than becoming an unlinked pronoun or being silently dropped.

KGGen, designed specifically for this stage, uses a language model-based extractor to predict subject-predicate-object triples and then applies an iterative clustering algorithm to refine the raw graph. Its design reflects a recognition that the sparse, disconnected graph produced by naive extraction is the common failure mode, not an edge case. The fact that someone built a dedicated tool to solve this specific problem is itself informative about how reliably the problem appears in practice.

Resolution quality predicts queryability more reliably than any other pipeline stage. A graph with a million correctly resolved, well-connected nodes is more useful for downstream reasoning than one with ten million ambiguous ones. Coherence matters more than size, and that ordering of priorities is not obvious until you have spent time debugging a graph that has plenty of the latter and very little of the former.

Stage Four: Choosing a Data Model and Schema Before Loading Anything

Two data models dominate production knowledge graph deployments, and the choice between them has consequences significant enough to warrant real deliberation before a single entity is loaded.

The Resource Description Framework, RDF, represents data as standardized triples stored in triple stores and queried via SPARQL. It is interoperable by design, queryable via SPARQL, and exchangeable in formats including N-Triples, Turtle, and JSON-LD. The semantic web ecosystem is built on it. If a graph needs to participate in Linked Open Data or interoperate with external ontologies, RDF is the natural default. Its limitation is expressive: a bare RDF triple cannot natively represent multiple distinct relationships of the same type between two nodes, so modeling that requires reification or named graphs, which add complexity and make queries noticeably more cumbersome under production load. RDF was designed for interoperability and reasoning, and that design priority becomes apparent quickly when the use case is something else entirely.

The Labeled Property Graph model, implemented most prominently by Neo4j and queryable via Cypher or the newer ISO-standard GQL, takes a different approach. Nodes and edges carry properties directly. A relationship between a person and an organization can carry an "employed since" date as a property of the edge itself, without reification. This makes the model more natural for the kinds of data most web extraction pipelines produce: organizational structures, product relationships, event timelines. The tradeoff is that LPG is not standardized across vendors the way RDF is and does not natively support the formal reasoning capabilities that RDF-based ontology languages provide.

The choice is largely a question of downstream use. Graphs powering application queries, recommendation systems, or retrieval-augmented generation workflows tend to fit LPG better. Graphs where semantic interoperability and formal inference are primary requirements tend to favor RDF.

Ontology design, defining the types of entities and relationships the graph will represent, is the second decision at this stage. It is consistently underestimated and consistently expensive when deferred. Retrofitting an ontology onto an already-loaded graph requires either reprocessing every entity or maintaining dual representations during a transition period. Recent research suggests GPT-4's ontology outputs approach the quality of novice human modelers when given well-formed competency questions, per a 2025 study available at arxiv.org/html/2510.20345v1. That matters for teams without ontology engineers on staff, though it does not eliminate the need for domain expert review. LLM-drafted ontologies reflect the model's prior understanding of a domain; they do not substitute for a practitioner's knowledge of which distinctions actually matter in context and which will cause problems at query time.

What happens when schema design is skipped is visible in any graph that has been incrementally loaded without governance: relation type counts grow without bound, query patterns become unreliable, and extending the schema to accommodate new entity types breaks existing queries in ways that surface gradually rather than all at once. This gets described, charitably, as organic growth.

Stage Five: Loading Into a Graph Database and Keeping It Current

Loading is where resolution and schema decisions become irreversible at scale. Resolved entities become nodes; normalized relations become edges with properties derived from the extraction and resolution stages. The mechanics are not especially complicated. What is complicated is cross-document entity matching at load time: an entity mentioned across ten distinct sources should resolve to one node, and that resolution must happen at ingestion, not as a post-hoc cleanup pass.

Neo4j's LLM Knowledge Graph Builder, which had become the fourth most popular source of user interaction on AuraDB Free as of early 2025, handles this by chunking documents, computing embeddings, extracting entities and relationships, and linking pieces spread across multiple sources. The implementation reflects a design assumption that multi-source ingestion is the default case for web-derived graphs, not a special case to be addressed later.

The maintenance problem is at least as significant as the initial load, and considerably more often neglected. Web data changes continuously. Companies are acquired, products are discontinued, executives move, facts are corrected. A graph built once and left unrefreshed degrades gradually, which makes the degradation easy to overlook until downstream consumers stop trusting the results. The split between initial population and ongoing update is not an operational detail; it is a structural requirement that must be designed into the pipeline architecture before the first entity is loaded, because the mechanisms that support efficient maintenance, provenance tracking, change detection, targeted re-extraction, are expensive to retrofit.

Provenance tracking is what makes maintenance tractable in practice. Each node and triple should carry its source URL and extraction timestamp. With that information, a change detection system, implementing what is broadly called change data capture, can identify which source pages have updated since last extraction, trigger targeted re-scraping and re-extraction for those pages only, and update affected entities without requiring a full rebuild. Without provenance, the only alternative when data quality degrades is to rebuild from scratch, which is expensive at small scale and essentially impossible at large scale.

Neo4j's partnership with Microsoft Azure, announced in 2024, reflects where production graph infrastructure is heading: integrated graph and generative AI capabilities on managed cloud platforms, targeting use cases where explainability and structured reasoning matter. The graph database layer is becoming less of a standalone component and more of a first-class participant in AI system architecture. One might argue that this integration genuinely simplifies the pipeline — but does it, or does it simply relocate the complexity elsewhere? That remains an open question worth watching before committing to any particular managed offering.

Where Each Stage Breaks and What the Failure Looks Like Downstream

The defining characteristic of pipeline failure in knowledge graph construction is that defects introduced early do not stay contained. They propagate forward and compound, often invisibly, until the graph is in use, at which point tracing an error back to its origin is painful work.

Extraction failures take two dominant forms. Missed dynamic content means entities that exist on the live web never enter the pipeline; the graph has no record of their absence, which is the most insidious kind of incompleteness because it cannot be detected from inside the graph. Boilerplate pollution is the opposite problem: navigation text, footer links, and ad copy pass downstream as substantive content, and the NLP stage extracts entities from them faithfully, producing nodes that represent page furniture rather than real-world things.

At the entity recognition and relation extraction stage, the characteristic failure is low recall on domain-specific entity types that fall outside the model's training distribution. A model that performs well on general organizational and person entities may miss the specialized vocabulary of a particular scientific domain or industry vertical entirely. And even when aggregate extraction accuracy is high, the relationship between accuracy rate and absolute error count is arithmetic: at sufficient scale, the absolute error volume entering the graph becomes non-trivial regardless of how good the rate looks on paper.

Entity resolution failures bifurcate into two modes with opposite effects. Under-merging leaves the graph fragmented: queries return no paths between entities that should be connected, and the graph appears sparser than the underlying data warrants. Over-merging is worse in a different way; distinct entities collapse into a single node, and every query that touches that node returns results that blend two or more real-world things together. Both failures require validation against ground truth to detect. Neither announces itself in the query results.

Schema failures accumulate more slowly and metastasize more thoroughly. Relation type explosion, the accumulation of dozens of semantically equivalent predicates with no normalization, means queries must be written defensively, enumerating every possible predicate variant to avoid missing relevant results. Schema drift, the addition of new data without alignment to the existing ontology, breaks existing queries silently: the queries return results, but the results no longer mean what the query author intended. This is particularly difficult to catch in systems without robust query-level testing.

Loading and maintenance failures close the loop. Stale facts are the most visible: the graph reflects the state of the web at ingestion time, and that state recedes from reality with every passing day. Missing provenance is the failure that prevents remediation. When a fact is wrong and there is no record of which source introduced it, targeted correction is impossible; the options are to trust a wrong fact, correct it manually without tracing its origin, or rebuild, and none of those options scales gracefully.

Each decision made at the design stage, what to extract, how to resolve, which schema to adopt, how to track provenance, shapes what is possible at every subsequent stage in ways that are genuinely difficult to unwind. The graphs that hold up at scale are not the ones where everything went smoothly. They are the ones where someone made those commitments deliberately, early, and with a clear-eyed view of what each stage's output needs to look like for the next stage to function at all.