Come work alongside our client's Data and AI teams as they stand up a concierge AI companion tool.
Key Area 1: the science behind NLP/LLMs
The critical gap to fill is the expertise in NLP's; in order to build out a GenAI companion that truly captures the wealth of information we have access to, the team needs an expert who understands how these algorithms function. This requires deep knowledge of NLP and machine learning, along with practical experience in both integrating pre-trained models and developing custom models tailored to specific needs.
Key Area 2: Architecture
A significant challenge for companion agents is being integrated with the context. Knowing how to architect the model to capture and utilize context effectively is essential to designing a companion that meets the task requirements. A Data Scientist with NLP/LLM expertise is crucial for determining when to leverage out-of-the-box solutions (like a generalized pre-trained model such as GPT4o) and when to develop custom applications using other state-of-the-art NLP models. This includes making strategic decisions on when to switch to the latest advancements versus selecting the best choice for the specific task.
Key Area 3: Hyperparameter Tuning, Feature Engineering, Evaluation
Closely related to the science behind these algorithms is the understanding of how to optimize them through hyperparameter tuning, feature engineering, and thorough evaluation. Achieving the best outcomes often requires tuning beyond default settings. This includes techniques like truncation, permutation importances, and other optimization strategies paired with the main model. It also involves conducting comprehensive evaluations and accurately interpreting the outputs to inform further improvements.