A Guide to Natural Language Processing With AllenNLP

About this guide

We walk through the basics of using AllenNLP, describing all of the main abstractions used and why we chose them, how to use specific functionality like configuration files or pre-trained representations, and how to build various kinds of models, from simple to complex.

Overview

This chapter will give an overview of AllenNLP, and will outline the main chapters of this guide

Begin Chapter

Explore the guide material

Part 1: Quick Start

Part 1 gives you a quick walk-through of main AllenNLP concepts and features. We’ll build a complete, working NLP model (text classifier) along the way.

Explore Part 1

Part 2: Abstractions, Design, and Testing

Part 2 provides in-depth tutorials on individual abstractions and features of AllenNLP.

Explore Part 2

Reading data

This chapter provides a deep dive into AllenNLP abstractions that are essential for reading data, including fields and instances, dataset readers, vocabulary, and how batching is handled in AllenNLP

Building your model

This chapter gives a deep dive into one of the most important components of AllenNLP—Model—and an in-depth guide to building your own model.

Common architectures

In this chapter we'll introduce neural architectures and AllenNLP abstractions that are commonly used for building your NLP model.

Representing text as features: Tokenizers, TextFields, and TextFieldEmbedders

A deep dive into AllenNLP's core abstraction: how exactly we represent textual inputs, both on the data side and the model side.

Writing tests for your code

(Coming soon) This chapter gives our recommendations for testing practices in NLP code, and describes the utilities that AllenNLP provides to make it easier.

Making predictions and serving demos

(Coming soon) This chapter describes how AllenNLP data and model code is set up to make it easy for you serve a demo of your model.

Using config files: FromParams and Registrable

This chapter describes AllenNLP's simple dependency injection framework.

Writing your own scripts

(Coming soon) Here we talk about how to easily use allennlp without using our built in commands, if you have more advanced needs or find it easier to reason about python code than configuration files.

Debugging your AllenNLP code

Some tips and tricks for using an IDE debugger with AllenNLP.

Hyperparameter Optimization

This chapter gives a basic tutorial for optimizing the hyperparameters of your model, using Optuna as an example.

Part 3: Practical Tasks With AllenNLP

Part 3 introduces common NLP tasks and how to build models for these tasks using AllenNLP.

Explore Part 3

A Guide to Natural Language Processing With AllenNLP

About this guide

Overview

Explore the guide material

Part 1: Quick Start

Your first model

Training and prediction

Next steps