Building materialised views for linked data systems using microservices
In the BBC’s Content Distribution Services division, we build and maintain systems that provide content metadata to a wide range of audience-facing products.
Our current architecture for distributing tagging metadata consists mainly of two RDF-based read and write APIs feeding off a central triplestore. This single storage setup for all operations imposes restrictions on performance and scalability.
I will talk about our work done to create an event-driven distribution pipeline that generates materialised views of tagging metadata.
The new microservices architecture comprises of small, single-purpose services, lambda functions, event stores, queues, streams etc. The views are built on data stores that are optimised to serve specific query profiles thus improving the overall performance and scalability of the system.
Augustine Kwanashie
Software Engineer, BBC
Video
Presentation