Hades/Olympus
1 Year (2026)
A LLM-powered web crawling solution designed to scrape and extract job listings from company websites for our clients. Leveraging an LLM for identification, extraction, and pagination tasks, the multi-country, multi-client system efficiently processed up to 5,000 websites daily. The architecture was built entirely in Python, utilizing tools within Palantir Foundry, including Pipelines, Datasets, and Workshop Applications.
Python
SQL
Palantir Foundry
Pipelines
Big Data
Jina
ScrapingBee
Scrapy
Prompts
LLM