Language Data Scientist, Amazon

Job ID: 2649441 | Amazon.com Services LLC

The Shopping Tech Foundation Team is looking for a Language Data Scientist to collaborate in developing solutions for LLM prompt engineering, LLM evaluation/benchmarking, and annotation efficiency. This position is an opportunity to apply your linguistic and data science expertise in a challenging but supportive environment.

Do you want to be part of the team developing the future technology that impacts the customer experience of ground-breaking products? Then come join us and make history.

Our team works on a variety of projects, including state of the art generative AI, LLM finetuning, alignment, prompt engineering, benchmarking solutions. We are customer obsessed and committed to delivering results with the highest quality and integrity.

As a Language Data Scientist, you will start by diving deep into a couple of critical LLM related projects. You will collaborate with fellow applied scientists, language data scientists, program managers, as well as stakeholders in engineering, annotation operation teams, and product teams to understand the role data plays in developing models that meet customer needs. You will analyze, follow, and improve processes for collecting, assessing and improving LLM inputs and outputs, and automating where appropriate.

You will apply state-of-the-art Generative AI techniques to analyze how well our data represents human language and run experiments to gauge downstream interactions. You will work collaboratively with other language data scientists and scientists to design and implement principled strategies for data optimization.

Key job responsibilities

1. Source, validate, and deliver high-quality language model artifacts, and linguistic data
2. Collaborate with stakeholders to design data collection and LLM development efforts
3. Oversee the progress and quality of several data collection, model development and annotation projects at a time
4. Advocate for strict adherence to data guidelines and quality thresholds
5. Extend existing data collection, annotation, and quality assurance efforts to support feature and language expansion
6. Innovate on data collection and LLM finetuning/prompt engineering methodologies, guidelines, quality metrics to support new requests
7. Automate repetitive workflows and improve existing processes

BASIC QUALIFICATIONS

1. 2+ years of data scientist experience
2. 3+ years of data querying languages (e.g. SQL), scripting languages (e.g. Python) or statistical/mathematical software (e.g. R, SAS, Matlab, etc.) experience
3. 3+ years of machine learning/statistical modeling data analysis tools and techniques, and parameters that affect their performance experience
4. Experience applying theoretical models in an applied environment
5. Master's degree in a quantitative field such as statistics, mathematics, data science, business analytics, economics, finance, engineering, or computer science

PREFERRED QUALIFICATIONS

1. Experience in Python, Perl, or another scripting language
2. Experience in a ML or data scientist role with a large technology company
3. Knowledge of relevant statistical measures such as confidence intervals, significance of error measurements, development and evaluation data sets, etc.

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit