This project showcases the application hosted at app.nbdata.co.
VectoredIn is a web application developed as a submission for the Weaviate Machine Learning Engineer challenge. It’s designed to analyze and visualize job market data using advanced natural language processing and machine learning techniques.
Key features include:
3D Plot of Semantic Distance: An interactive visualization that allows users to explore job listings in a three-dimensional space based on user-defined semantic axes.
RAG (Retrieval-Augmented Generation) Search: Enhances large language model queries by providing additional context from the job data.
Plot Summary: Generates a summary of job listings, highlighting similarities and differences using Weaviate’s generative search and OpenAI’s language model.
Axis Alignment: Allows users to select specific points in the 3D scatter plot and view a detailed comparison of how well the selected job aligns with the three chosen axes.
HNSW & ANN Implementation: Utilizes the Hierarchical Navigable Small World algorithm for efficient similarity search, with custom modifications to return a range of results with varying levels of similarity.
The project is built using Python, Django, and JavaScript, with Plotly for visualizations and Weaviate for vector storage and retrieval. It demonstrates advanced techniques in semantic search, data visualization, and natural language processing applied to job market analysis.
GrainCorp
2021-2023
Graincorp is a leading agribusiness, specializing in grain and edible oil storage and handling. During tenure, innovative data solutions for various operational challenges were introduced. Key initiatives included automating data processes to enhance the seasonal hiring workflow and developing real-time dashboards for harvest management, thereby driving significant efficiency and decision-making improvements within the company.
Pioneered automation of manual reporting, reducing team workload.
Designed and implemented live dashboards for management metrics.
Formulated new KPIs, improving task completion rates significantly.
Transformed HR’s seasonal hiring with an automated tracking system for over 10,000 applicants.
Kaggle
2020-Present
The Kaggle LuxAI Challenge offered a platform to harness AI for strategizing in an adversarial gaming setting, promoting innovative thinking and sharp data science skills. The challenge demanded quick iteration and clever algorithm application to create effective game strategies and provided a collaborative environment to compete with global participants, enriching strategic problem-solving abilities.
Tackled complex strategic AI applications within a competitive setting.
Innovatively applied AI algorithms to game scenarios through the LuxAI Challenge.
Showcased agility in developing solutions against stringent competitive constraints.
Strengthened machine learning acumen in an intellectually stimulating community.
VectoredIn is a tool I developed to visualize the job market and job postings from LinkedIn utilising various tools and techniques from the NLP and LLM space. To read more about the project, look here.
Data For this project, I built upon this Kaggle dataset here, which originally contained over 100,000 job postings, and I increased this to just over 1 Million individual job postings.
With each job posting having an average of 500 words (600 tokens) for the description alone, alongside some additional metadata, this is a reasonably large dataset.
Read more
Open to opportunities in the data technology industry. My availability extends to both contracting roles and project-based work. I am also open to full-time positions, provided the role aligns with my professional goals and interests.
Please feel free to reach out to me with any inquries.
