A simple setup to perform a breadth-first crawl of a given web-domain, storing per page (meta)data such as internal and external links etc and semantically split chunks with embeddings for consumption of a RAG applications.
A simple setup to perform a breadth-first crawl of a given web-domain, storing per page (meta)data such as internal and external links etc and semantically split chunks with embeddings for consumption of a RAG applications.
Build internal AI tools, chatbots, or content analysis with up-to-date, searchable knowledge. Use the open-source stack of Crawl4AI, Supabase, and n8n for building Retrieval-Augmented Generation workflows.
Read about our thoughts, projects and ideas.
