Today we’re joined by Stephen Merity, an independent researcher focused on NLP and Deep Learning. In our conversation, we discuss Stephens latest paper, Single Headed Attention RNN: Stop Thinking With Your Head, detailing his primary motivations behind the paper, the decision to use SHA-RNNs for this research, how he built and trained the model, his approach to benchmarking, and finally his goals for the research in the broader research community.
In the inaugural TWiML Live, Sam Charrington is joined by Amanda Askell (OpenAI), Anima Anandkumar (NVIDIA/CalTech), Miles Brundage (OpenAI), Robert Munro (Lilt), and Stephen Merity to discuss the controversial recent release of the OpenAI GPT-2 Language Model.
We cover the basics like what language models are and why they’re important, and why this announcement caused such a stir, and dig deep into why the lack of a full release of the model raised concerns for so many.
