Research Summaries and Trends

• NLP-Overview is an up-to-date overview of deep learning techniques applied to NLP, including theory, implementations, applications, and state-of-the-art results. This is a great Deep NLP Introduction for researchers.

• NLP-Progress tracks the progress in Natural Language Processing, including the datasets and the current state-of-the-art for the most common NLP tasks

• NLP's ImageNet moment has arrived

• ACL 2018 Highlights: Understanding Representation and Evaluation in More Challenging Settings

• Four deep learning trends from ACL 2017. Part One: Linguistic Structure and Word Embeddings

• Four deep learning trends from ACL 2017. Part Two: Interpretability and Attention

• Highlights of EMNLP 2017: Exciting Datasets, Return of the Clusters, and More!

• Deep Learning for Natural Language Processing (NLP): Advancements & Trends

• Survey of the State of the Art in Natural Language Generation

Prominent NLP Research Labs

Back to Top

• The Berkeley NLP Group - Notable contributions include a tool to reconstruct long dead languages, referenced here and by taking corpora from 637 languages currently spoken in Asia and the Pacific and recreating their descendant.

• Language Technologies Institute, Carnegie Mellon University - Notable projects include Avenue Project, a syntax driven machine translation system for endangered languages like Quechua and Aymara and previously, Noah's Ark which created AQMAR to improve NLP tools for Arabic.

• NLP research group, Columbia University - Responsible for creating BOLT ( interactive error handling for speech translation systems) and an un-named project to characterize laughter in dialogue.

• The Center or Language and Speech Processing, John Hopkins University - Recently in the news for developing speech recognition software to create a diagnostic test or Parkinson's Disease, here.

• Computational Linguistics and Information Processing Group, University of Maryland - Notable contributions include Human-Computer Cooperation or Word-by-Word Question Answering and modeling development of phonetic representations.

• Penn Natural Language Processing, University of Pennsylvania- Famous for creating the Penn Treebank.

• The Stanford Nautral Language Processing Group- One of the top NLP research labs in the world, notable for creating Stanford CoreNLP and their coreference resolution system

Tutorials

Back to Top

Reading Content

General Machine Learning

• Machine Learning 101 from Google's Senior Creative Engineer explains Machine Learning for engineer's and executives alike

• AI Playbook - a16z AI playbook is a great link to forward to your managers or content for your presentations

• Ruder's Blog by Sebastian Ruder for commentary on the best of NLP Research

• How To Label Data guide to managing larger linguistic annotation projects

• Depends on the Definition collection of blog posts covering a wide array of NLP topics with detailed implementation

Introductions and Guides to NLP

• Understand & Implement Natural Language Processing

• NLP in Python - Collection of Github notebooks

• Natural Language Processing: An Introduction - Oxford

• Deep Learning for NLP with Pytorch

• Hands-On NLTK Tutorial - NLTK Tutorials, Jupyter notebooks

• Natural Language Processing with Python – Analyzing Text with the Natural Language Toolkit - An online and print book introducing NLP concepts using NLTK. The book's authors also wrote the NLTK library.

• Train a new language model from scratch - Hugging Face 🤗

• The Super Duper NLP Repo (SDNLPR): Collection of Colab notebooks covering a wide array of NLP task implementations.

Blogs and Newsletters

• Deep Learning, NLP, and Representations

• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) and The Illustrated Transformer

• Natural Language Processing by Hal Daumé III

• arXiv: Natural Language Processing (Almost) from Scratch

• Karpathy's The Unreasonable Effectiveness of Recurrent Neural Networks

• Machine Learning Mastery: Deep Learning for Natural Language Processing

• Visual NLP Paper Summaries

Videos and Online Courses

Back to Top

• Advanced Natural Language Processing - CS 685, UMass Amherst CS

• Deep Natural Language Processing - Lectures series from Oxford

• Deep Learning for Natural Language Processing (cs224-n) - Richard Socher and Christopher Manning's Stanford Course

• Neural Networks for NLP - Carnegie Mellon Language Technology Institute there

• Deep NLP Course by Yandex Data School, covering important ideas from text embedding to machine translation including sequence modeling, language models and so on.

• fast.ai Code-First Intro to Natural Language Processing - This covers a blend of traditional NLP topics (including regex, SVD, naive bayes, tokenization) and recent neural network approaches (including RNNs, seq2seq, GRUs, and the Transformer), as well as addressing urgent ethical issues, such as bias and disinformation. Find the Jupyter Notebooks here

• Machine Learning University - Accelerated Natural Language Processing - Lectures go from introduction to NLP and text processing to Recurrent Neural Networks and Transformers. Material can be found here.

• Applied Natural Language Processing- Lecture series from IIT Madras taking from the basics all the way to autoencoders and everything. The github notebooks for this course are also available here

Books

• Speech and Language Processing - free, by Prof. Dan Jurafsy

• Natural Language Processing - free, NLP notes by Dr. Jacob Eisenstein at GeorgiaTech

• NLP with PyTorch - Brian & Delip Rao

• Text Mining in R

• Natural Language Processing with Python

• Practical Natural Language Processing

• Natural Language Processing with Spark NLP

• Deep Learning for Natural Language Processing by Stephan Raaijmakers

• Real-World Natural Language Processing - by Masato Hagiwara

• Natural Language Processing in Action, Second Edition - by Hobson Lane and Maria Dyshel