Share this article

With KB5043178 to Release Preview Channel, Microsoft advises Windows 11 users to plug in when the battery is low

Copilot in Outlook will generate personalized themes for you to customize the app

Microsoft will raise the price of its 365 Suite to include AI capabilities

Death Stranding Director’s Cut is now Xbox X|S at a huge discount

Outlook will let users create custom account icons so they can tell their accounts apart easier

The largest powerful language model trained by Microsoft and NVIDIA

2 min. read

Published onOctober 12, 2021

published onOctober 12, 2021

Share this article

Read our disclosure page to find out how can you help Windows Report sustain the editorial teamRead more

Key notes

Microsoft and NVIDIA have today announced that they have successfully trained the largest most powerful language to date. The Megatron-Turing Natural Language Generation (MT-NLP) is meant to be the successor to the companies’ Turing NLG 17B and Megatron-LM models.

The MT-NLP has 530 billion parameters with the capability of a wide set of natural language tasks. According to the two companies, it also has comprehension, reasoning and natural language capabilities.

First breakthrough

The two companies have in the past worked on several innovations but this is considered the most powerful.

The quality obtained is a significant step towards the journey of unlocking AI in natural language. The two innovations DeepSpeed and Megatron-LM will be the major beneficiaries of the AI model development and open the pathway for large AI models to be affordable and faster to train.

Microsoft trains a 530billion parameter GPT3-style language model. This is the largest LM in existence. (There’s also the mysterious multi-modal 1.5trillion+ ‘Wu Dao’ MOE model but little known about it). Microsoft trains on ‘The Pile’ dataset.https://t.co/md03QzqlxA

Training

The training took place across560 Nvidia DGX A100 servers, with 8 Nvidia A100 80GB GPUs for each.

Although the MT-NLP has the capability of inferring basic mathematical operations, it is not entirely accurate. It however surpasses memorization and can complete tasks.

Such models are crucial in amplifying the biases present in data in which they are trained.

Although Microsoft acknowledges there have been challenges, they are committed to addressing them by making continuous milestones through continued research while minimizing potential harm to users.

For now, users can enjoy the milestones made as we wait to see what is next in store.

What are your thoughts on the collaboration between Microsoft and NVIDIA? DO you have any expectations? Let us know in the comment section below.

Don Sharpe

Tech Journalist

Don has been writing professionally for over 10 years now, but his passion for the written word started back in his elementary school days. His work has been published on Livebitcoinnews.com, Learnbonds.com, eHow, AskMen.com, Forexminute.com, The Writers Network and a host of other companies.

User forum

0 messages

Sort by:LatestOldestMost Votes

Comment*

Name*

Email*

Commenting as.Not you?

Save information for future comments

Comment

Don Sharpe

Tech Journalist

Don has been writing professionally for over 10 years now, simplifying the tech universe for the mases.

With KB5043178 to Release Preview Channel, Microsoft advises Windows 11 users to plug in when the battery is low#

Copilot in Outlook will generate personalized themes for you to customize the app#

Microsoft will raise the price of its 365 Suite to include AI capabilities#

Death Stranding Director’s Cut is now Xbox X|S at a huge discount#

Outlook will let users create custom account icons so they can tell their accounts apart easier#

The largest powerful language model trained by Microsoft and NVIDIA#